Smojo Developer Group Batch #4

  1. #1: Kickoff; the data transformation approach to solving problems.
  2. #2: Review of Ch6 exercises; unique words calculator; coding guidelines.
  3. #3: PARSE. Writing Stack and Line Comments, TICK and BRACKET-TICK. Usage of XTs.
  4. #4: Metaprogramming Example: Automatically writing Getters & Setters.
  5. #5: IMMEDIATE, POSTPONE, LATESTXT, COMPILATION?; Creating BRACKET-TICK and CONSTANT.
  6. #6: Constants vs Variables vs Locals. Redoing m{ ... }m. Busby Walkthrough.
  7. #7: Implementing VAR. The spare stack. JSON explanation. NASA/JPL API Walkthrough.
  8. #8: Explanation and simplification of %. JSON explained. Using JSON>. Closest Approach API used.
  9. #9: Modules, Profiler & A solution for Closest Objects.
  10. #10: Vectors and Embeddings.
  11. #11: Vectors and Embeddings (Answers to Homework)
  12. #12: Intro to Smojo VM
  13. #13: Servers Part 1
  14. #14: Answers to Homework Session #13
  15. #15: Servers Part 2: Blocking vs Non-Blocking I/O; Single vs Multi-Threading.
  16. #16: Project YT Summarizer template, downloading transcripts.
  17. #17: Submitting YT links, saving to memory.
  18. #18: Creating Listings, Saving to Disk using Phashes.
  19. #19: Autoloading the listing, Embedding Server for testing, Beginning Chunking.
  20. #20: UI Improvements,Chunk Retrieval Algorithm, Creating & Saving Chunks.
  21. #21: Adding UI for making a query, Connecting to Chunk Retrieval and prompt generation.
  22. #22: Making a VM library to connect to OpenAI's Embedding & GPT.
  23. #23: Completing connector to GPT, Synchronous vs Asynchronous Communication.
  24. #24: Responder vs Caller Asynchronous Communication, Completing Async embeddings for chunks.
  25. #25: Hookup to GPT, tidying up the UI, Async gpt query, basic tests.

Session #25 marks our final session. The code for LLM and YTSum servers as well as the OpenAI connector are all on my community page.

Where do you go from here?

First, get the system to work for you. Then think about making increasing challenging changes. Eg:

  1. Tidy up the UI further so that is becomes more usable. Eg, make the query disappear after it has been submitted
  2. Make the number of chunks a user-definable parameter. What is the consequence of allowing a user select this randomly?
  3. Predefined queries: A set of queries that would apply to all transcripts. Use buttons maybe? User clicks and gets the answer.
  4. Allow analysing multiple transcripts -- useful to allow the user to query multiple videos, not just one
  5. Store previous queries and emulate the "ChatGPT" display.
  6. What happens if something goes wrong? Does the system fail gracefully? You need to poke around to "break" the servers. Should not be to hard. But also document what happened to break it. Eg, giving a VID that does not exist or does not have a transcript.
As you work on these improvements, you will also grow as a programmer. As always, feel free to ask over Telegram. These videos will be up for as long as you need them.

Smojo Developer Group Batch #3

YTSum Videos

  1. Session Management (Slides)
  2. The Message Queue (Slides)
  3. Deploying Microservices (Slides)
  4. Downloading, Parsing JSON, YT Transcripts (Slides)

Singapore

Indonesia