Migrate from 0.9 to 1.0

Genkit 1.0 introduces many feature enhancements that improve overall functionality; it also has some breaking changes. If you have been developing applications with Genkit 0.9, you need to update your application code when you upgrade to the latest version of Genkit. This guide outlines the most significant changes, and explains how to migrate your existing applications smoothly.

Beta APIs

We're introducing an unstable, Beta API channel, and leaving session, chat and Genkit client APIs in beta as we continue to refine them. More specifically, the following functions are currently in the beta namespace:

  • ai.chat
  • ai.createSession
  • ai.loadSession
  • ai.currentSession
  • ai.defineFormat
  • ai.defineInterrupt

Old:

import { genkit } from 'genkit';
const ai = genkit({...})
const session = ai.createSession({ ... })

New:

import { genkit } from 'genkit/beta';
const ai = genkit({...})
const session = ai.createSession({ ... })

Old:

import { runFlow, streamFlow } from 'genkit/client';

New:

import { runFlow, streamFlow } from 'genkit/beta/client';

Introducing new @genkit-ai/express package

This new package contains utilities to make it easier to build an Express.js server with Genkit. You can find more details about this on this page.

startFlowServer has moved from part of the genkit object to this new @genkit-ai/express package; to use startFlowServer, you must update your imports.

Old:

const ai = genkit({ ... });
ai.startFlowServer({
  flows: [myFlow1, myFlow2],
});

New:

import { startFlowServer } from '@genkit-ai/express';
startFlowServer({
  flows: [myFlow1, myFlow2],
});

Changes to Flows

There are several changes to flows in 1.0:

  • ai.defineStreamingFlow has been consolidated into ai.defineFlow,
  • onFlow has been replaced by onCallGenkit,
  • run has moved to ai.run,
  • There are changes to working with auth.

The run function for custom trace blocks has moved to part of the genkit object; use ai.run to invoke it instead.

Old:

ai.defineFlow({name: 'banana'}, async (input) => {
  const step = await run('myCode', async () => {
    return 'something'
  });
})

New:

ai.defineFlow({name: 'banana'}, async (input) => {
  const step = await ai.run('myCode', async () => {
    return 'something'
  });
})

ai.defineStreamingFlow has been removed; use ai.defineFlow instead. Also, streamingCallback has moved to a field inside the second argument of the flow function and is now called sendChunk.

Old:

const flow = ai.defineStreamingFlow({name: 'banana'}, async (input, streamingCallback) => {
  streamingCallback({chunk: 1});
})

const {stream} = await flow()
for await (const chunk of stream) {
  // ...
}

New:

const flow = ai.defineFlow({name: 'banana'}, async (input, {context, sendChunk}) => {
  sendChunk({chunk: 1});
})

const {stream, output} = flow.stream(input);
for await (const chunk of stream) {
  // ...
}

FlowAuth auth is now called context. You can access auth as a field inside context:

Old:

ai.defineFlow({name: 'banana'}, async (input) => {
  const auth = getFlowAuth();
  // ...
})

New:

ai.defineFlow({name: 'banana'}, async (input, { context }) => {
  const auth = context.auth;
})

onFlow moved to firebase-functions/https package and has been renamed to onCallGenkit. The following snippet shows an example of how to use it.

Old

import { onFlow } from "@genkit-ai/firebase/functions";

export const generatePoem = onFlow(
  ai,
  {
    name: "jokeTeller",
    inputSchema: z.string().nullable(),
    outputSchema: z.string(),
    streamSchema: z.string(),
  },
  async (type, streamingCallback) => {
    const { stream, response } = await ai.generateStream(
      `Tell me a longish ${type ?? "dad"} joke.`
    );
    for await (const chunk of stream) {
      streamingCallback(chunk.text);
    }
    return (await response).text;
  }
);

New:

import { onCallGenkit } from "firebase-functions/https";
import { defineSecret } from "firebase-functions/params";
import { genkit, z } from "genkit";

const apiKey = defineSecret("GOOGLE_GENAI_API_KEY");

const ai = genkit({
  plugins: [googleAI()],
  model: gemini15Flash,
});

export const jokeTeller = ai.defineFlow(
  {
    name: "jokeTeller",
    inputSchema: z.string().nullable(),
    outputSchema: z.string(),
    streamSchema: z.string(),
  },
  async (type, { sendChunk }) => {
    const { stream, response } = ai.generateStream(
      `Tell me a longish ${type ?? "dad"} joke.`
    );
    for await (const chunk of stream) {
      sendChunk(chunk.text);
    }
    return (await response).text;
  }
);

export const tellJoke = onCallGenkit({ secrets: [apiKey] }, jokeTeller);

Auth policies have been removed from defineFlow. Handling of auth policies is now server-dependent.

Old:

export const simpleFlow = ai.defineFlow(
  {
    name: 'simpleFlow',
    authPolicy: (auth, input) => {
      // auth policy
    },
  },
  async (input) => {
    // Flow logic here...
  }
);

The following snippet shows an example of handling auth in Express.

New:

import { UserFacingError } from 'genkit';
import { ContextProvider, RequestData } from 'genkit/context';
import { expressHandler, startFlowServer } from '@genkit-ai/express';

const context: ContextProvider<Context> = (req: RequestData) => {
  return {
    auth: parseAuthToken(req.headers['authorization']),
  };
};

export const simpleFlow = ai.defineFlow(
  {
    name: 'simpleFlow',
  },
  async (input, { context }) => {
    if (!context.auth) {
      throw new UserFacingError("UNAUTHORIZED", "Authorization required.");
    }
    if (input.uid !== context.auth.uid) {
      throw new UserFacingError("UNAUTHORIZED", "You may only summarize your own profile data.");
    }
    // Flow logic here...
  }
);

const app = express();
app.use(express.json());
app.post(
  '/simpleFlow',
  expressHandler(simpleFlow, { context })
);
app.listen(8080);

// or

startFlowServer(
  flows: [withContextProvider(simpleFlow, context)],
  port: 8080
);

For more details, refer to the auth documentation.

The following snippet shows an example of handling auth in Cloud Functions for Firebase:

import { genkit } from 'genkit';
import { onCallGenkit } from 'firebase-functions/https';

const ai = genkit({ ... });;

const simpleFlow = ai.defineFlow({
  name: 'simpleFlow',
}, async (input) => {
  // Flow logic here...
});

export const selfSummary = onCallGenkit({
  authPolicy: (auth, data) => auth?.token?.['email_verified'] && auth?.token?.['admin'],
}, simpleFlow);

Prompts

We've made several changes and improvements to prompts.

You can define separate templates for prompt and system messages:

const hello = ai.definePrompt({
  name: 'hello',
  system: 'talk like a pirate.',
  prompt: 'hello {{ name }}',
  input: {
    schema: z.object({
      name: z.string()
    })
  }
});
const { text } = await hello({name: 'Genkit'});

Alternatively, you can define multi-message prompts in the messages field:

const hello = ai.definePrompt({
  name: 'hello',
  messages: '{{ role "system" }} talk like a pirate. {{ role "user" }} hello {{ name }}',
  input: {
    schema: z.object({
      name: z.string()
    })
  }
});

Instead of prompt templates you can use a function:

ai.definePrompt({
  name: 'hello',
  prompt: async (input, { context }) => {
    return `hello ${input.name}`
  },
  input: {
    schema: z.object({
      name: z.string()
    })
  }
});

You can access the context (including auth information) from within the prompt:

const hello = ai.definePrompt({
  name: 'hello',
  messages: 'hello {{ @auth.email }}',
});

Streaming functions do not require an await

Old:

const { stream, response } = await ai.generateStream(`hi`);
const { stream, output } = await myflow.stream(`hi`);

New:

const { stream, response } = ai.generateStream(`hi`);
const { stream, output } = myflow.stream(`hi`);

Embed has a new return type

We've added support for multimodal embeddings. Instead of returning just a single embedding vector, Embed returns an array of embedding objects, each containing an embedding vector and metadata.

Old:

const response = await ai.embed({embedder, content, options});  // returns number[]

New:

const response = await ai.embed({embedder, content, options}); // returns Embedding[]
const firstEmbeddingVector = response[0].embedding;  // is number[]