Flashcard generator with Instructor + Burr

Learn to use LLM structured outputs to create useful agents.

and

Nov 12, 2024

Flashcards help break down complex topics and learn anything from biology to a new language or lines for a play. This blog will show how to use LLMs to generate flashcards and kickstart your learning!

Instructor lets us get structured outputs from LLMs reliably, and Burr helps create an LLM application that's easy to understand and debug. It comes with Burr UI, a free, open-source, and local-first tool for observability, annotations, and more!

This post is a cross-post from a guest post on the Instructor blog and expands on an earlier one: Analyzing Youtube Transcripts with Instructor.

🤖 The full code example is available on GitHub

Generate flashcards using LLMs with Instructor

pip install openai instructor pydantic youtube_transcript_api "burr[start]"

1. Define the LLM response model

With instructor, you define Pydantic models that will serve as template for the LLM to fill.

Here, we define the QuestionAnswer model which will store the question, the answer, and some metadata. Attributes without a default value will be generated by the LLM.

This example shows several instructor features:

ue to prevent the LLM from hallucinating the value
- id generates a unique id (uuid)
The type annotation SkipJsonSchema also prevents the LLM from generating the value.
- youtube_url is set programmatically in the application. We don't want the LLM to hallucinate it.
Field can set constraints on what the LLM generates.
- min_items=3, max_items=5 to limit the number of potential answers between 3 and 5
- ge=0, lt=5 to limit the difficulty between 0 and 5 with 5 being the most difficult

2. Retrieve the YouTube transcript

We use youtube-transcript-api to get the full transcript of a video.

Extract the transcript of a YouTube video

Generate question-answer pairs

Now, to produce question-answer pairs:

Create an instructor client by wrapping the OpenAI client
Use .create_iterable() on the instructor_client to generate multiple outputs from the input
Specify response_model=QuestionAnswer to ensure outputs are QuestionAnswer objects
Use the messages to pass the task instructos via the system message, and the input transcript via user message.

This will return an generator that you can iterate over to access individual QuestionAnswer objects.

QuestionAnswer object generated by the LLM using Instructor

Create a flashcard application with Burr

Burr uses actions and transitions to define complex applications while preserving the simplicity of a flowchart for understanding and debugging.

1. Define `actions`

Actions are what your application can do. The @action decorator specifies what values can be read from or written to State. The decorated function takes a State as first argument and return an updated State object.

Next, we define three actions:

Process the user input to get the YouTube URL
Get the YouTube transcript associated with the URL
Generate question-answer pairs for the transcript

Note that this is only a light refactor from the previous code snippets.

Define 3 actions to get the user input, retrieve the YouTube transcript, and generate Q&A pairs.

2. Build the `Application`

To create a Burr Application, we use the ApplicationBuilder object.

Minimally, it needs to:

Use .with_actions() to define all possible actions. Simply pass the functions decorated with @action.
Use .with_transitions() to define possible transitions between actions. This is done via tuples (from_action, to_action).
Use .with_entrypoint() to specify which action to run first.

Build an Application from actions and transitions

Burr application graph — Visualization of the created Burr Application

3. Launch the application

Using Application.run() will make the application execute actions until a halt condition. In this case, we halt before process_user_input to get the YouTube URL from the user.

The method .run() returns a tuple (action_name, result, state). In this case, we only use the state to inspect the generated question-answer pairs.

Launch the Application using the `.run()` method

You can create a simple local experience by using .run() in a while loop

Next steps

Now that you know how to use Instructor for reliable LLM outputs and Burr to structure your application, many avenues open up depending on your goals!

1. Build complex agents

Instructor improves the LLM's reasoning by providing structure. Nesting models and adding constraints allow to get facts with citations or extract a knowledge graph in a few lines of code. Also, retries enable the LLM to self-correct.

Burr sets the boundaries between users, LLMs, and the rest of your system. You can add Condition on transitions to create complex workflows that remain easy to reason about.

2. Add Burr to your product

Your Burr Application is a lightweight Python object. You can run it within a notebook, via script, a web app (Streamlit, Gradio, etc.), or as a web service (e.g., FastAPI).

The ApplicationBuilder provides many features to productionize your app:

Persistence: save and restore State (e.g., store conversation history)
Observability: log and monitor application telemetry (e.g., LLM calls, number of tokens used, errors and retries)
Streaming and async: create snappy user interfaces by streaming LLM responses and running actions asynchronously.

For example, you can log telemetry into Burr UI in a few lines of code. First, instrument the OpenAI library. Then, add .with_tracker() the ApplicationBuilder with a project name and enabling use_otel_tracing=True.

Enable OpenTelemetry via the ApplicationBuilder

3. Annotate application logs

Burr UI has a built-in annotation tool that allows you to label, rate, or comment on logged data (e.g., user input, LLM response, content retrieved for RAG). This can be useful to create test cases and evaluation datasets.

Conclusion

We've shown how Instructor helps getting reliable outputs from LLMs and Burr provides the right tools to build an application. Now it's your turn to start building!