Genkit 1.0 introduces many feature enhancements that improve overall functionality; it also has some breaking changes. If you have been developing applications with Genkit 0.9, you need to update your application code when you upgrade to the latest version of Genkit. This guide outlines the most significant changes, and explains how to migrate your existing applications smoothly.
Beta APIs
We're introducing an unstable, Beta API channel, and leaving session, chat and Genkit client APIs in beta as we continue to refine them. More specifically, the following functions are currently in the beta
namespace:
ai.chat
ai.createSession
ai.loadSession
ai.currentSession
ai.defineFormat
ai.defineInterrupt
Old:
import { genkit } from 'genkit';
const ai = genkit({...})
const session = ai.createSession({ ... })
New:
import { genkit } from 'genkit/beta';
const ai = genkit({...})
const session = ai.createSession({ ... })
Old:
import { runFlow, streamFlow } from 'genkit/client';
New:
import { runFlow, streamFlow } from 'genkit/beta/client';
Introducing new @genkit-ai/express
package
This new package contains utilities to make it easier to build an Express.js server with Genkit. You can find more details about this on this page.
startFlowServer
has moved from part of the genkit object to this new
@genkit-ai/express
package; to use startFlowServer, you must
update your imports.
Old:
const ai = genkit({ ... });
ai.startFlowServer({
flows: [myFlow1, myFlow2],
});
New:
import { startFlowServer } from '@genkit-ai/express';
startFlowServer({
flows: [myFlow1, myFlow2],
});
Changes to Flows
There are several changes to flows in 1.0:
ai.defineStreamingFlow
has been consolidated intoai.defineFlow
,onFlow
has been replaced byonCallGenkit
,run
has moved toai.run
,- There are changes to working with auth.
The run
function for custom trace blocks has moved to part of the genkit
object; use ai.run
to invoke it instead.
Old:
ai.defineFlow({name: 'banana'}, async (input) => {
const step = await run('myCode', async () => {
return 'something'
});
})
New:
ai.defineFlow({name: 'banana'}, async (input) => {
const step = await ai.run('myCode', async () => {
return 'something'
});
})
ai.defineStreamingFlow
has been removed; use ai.defineFlow
instead. Also,
streamingCallback
has moved to a field inside the second argument of the flow
function and is now called sendChunk
.
Old:
const flow = ai.defineStreamingFlow({name: 'banana'}, async (input, streamingCallback) => {
streamingCallback({chunk: 1});
})
const {stream} = await flow()
for await (const chunk of stream) {
// ...
}
New:
const flow = ai.defineFlow({name: 'banana'}, async (input, {context, sendChunk}) => {
sendChunk({chunk: 1});
})
const {stream, output} = flow.stream(input);
for await (const chunk of stream) {
// ...
}
FlowAuth auth is now called context. You can access auth as a field inside context:
Old:
ai.defineFlow({name: 'banana'}, async (input) => {
const auth = getFlowAuth();
// ...
})
New:
ai.defineFlow({name: 'banana'}, async (input, { context }) => {
const auth = context.auth;
})
onFlow
moved to firebase-functions/https
package and has been renamed to
onCallGenkit
. The following snippet shows an example of how to use it.
Old
import { onFlow } from "@genkit-ai/firebase/functions";
export const generatePoem = onFlow(
ai,
{
name: "jokeTeller",
inputSchema: z.string().nullable(),
outputSchema: z.string(),
streamSchema: z.string(),
},
async (type, streamingCallback) => {
const { stream, response } = await ai.generateStream(
`Tell me a longish ${type ?? "dad"} joke.`
);
for await (const chunk of stream) {
streamingCallback(chunk.text);
}
return (await response).text;
}
);
New:
import { onCallGenkit } from "firebase-functions/https";
import { defineSecret } from "firebase-functions/params";
import { genkit, z } from "genkit";
const apiKey = defineSecret("GOOGLE_GENAI_API_KEY");
const ai = genkit({
plugins: [googleAI()],
model: gemini15Flash,
});
export const jokeTeller = ai.defineFlow(
{
name: "jokeTeller",
inputSchema: z.string().nullable(),
outputSchema: z.string(),
streamSchema: z.string(),
},
async (type, { sendChunk }) => {
const { stream, response } = ai.generateStream(
`Tell me a longish ${type ?? "dad"} joke.`
);
for await (const chunk of stream) {
sendChunk(chunk.text);
}
return (await response).text;
}
);
export const tellJoke = onCallGenkit({ secrets: [apiKey] }, jokeTeller);
Auth policies have been removed from defineFlow
. Handling of auth policies
is now server-dependent.
Old:
export const simpleFlow = ai.defineFlow(
{
name: 'simpleFlow',
authPolicy: (auth, input) => {
// auth policy
},
},
async (input) => {
// Flow logic here...
}
);
The following snippet shows an example of handling auth in Express.
New:
import { UserFacingError } from 'genkit';
import { ContextProvider, RequestData } from 'genkit/context';
import { expressHandler, startFlowServer } from '@genkit-ai/express';
const context: ContextProvider<Context> = (req: RequestData) => {
return {
auth: parseAuthToken(req.headers['authorization']),
};
};
export const simpleFlow = ai.defineFlow(
{
name: 'simpleFlow',
},
async (input, { context }) => {
if (!context.auth) {
throw new UserFacingError("UNAUTHORIZED", "Authorization required.");
}
if (input.uid !== context.auth.uid) {
throw new UserFacingError("UNAUTHORIZED", "You may only summarize your own profile data.");
}
// Flow logic here...
}
);
const app = express();
app.use(express.json());
app.post(
'/simpleFlow',
expressHandler(simpleFlow, { context })
);
app.listen(8080);
// or
startFlowServer(
flows: [withContextProvider(simpleFlow, context)],
port: 8080
);
For more details, refer to the auth documentation.
The following snippet shows an example of handling auth in Cloud Functions for Firebase:
import { genkit } from 'genkit';
import { onCallGenkit } from 'firebase-functions/https';
const ai = genkit({ ... });;
const simpleFlow = ai.defineFlow({
name: 'simpleFlow',
}, async (input) => {
// Flow logic here...
});
export const selfSummary = onCallGenkit({
authPolicy: (auth, data) => auth?.token?.['email_verified'] && auth?.token?.['admin'],
}, simpleFlow);
Prompts
We've made several changes and improvements to prompts.
You can define separate templates for prompt and system messages:
const hello = ai.definePrompt({
name: 'hello',
system: 'talk like a pirate.',
prompt: 'hello {{ name }}',
input: {
schema: z.object({
name: z.string()
})
}
});
const { text } = await hello({name: 'Genkit'});
Alternatively, you can define multi-message prompts in the messages field:
const hello = ai.definePrompt({
name: 'hello',
messages: '{{ role "system" }} talk like a pirate. {{ role "user" }} hello {{ name }}',
input: {
schema: z.object({
name: z.string()
})
}
});
Instead of prompt templates you can use a function:
ai.definePrompt({
name: 'hello',
prompt: async (input, { context }) => {
return `hello ${input.name}`
},
input: {
schema: z.object({
name: z.string()
})
}
});
You can access the context (including auth information) from within the prompt:
const hello = ai.definePrompt({
name: 'hello',
messages: 'hello {{ @auth.email }}',
});
Streaming functions do not require an await
Old:
const { stream, response } = await ai.generateStream(`hi`);
const { stream, output } = await myflow.stream(`hi`);
New:
const { stream, response } = ai.generateStream(`hi`);
const { stream, output } = myflow.stream(`hi`);
Embed has a new return type
We've added support for multimodal embeddings. Instead of returning just a single embedding vector, Embed returns an array of embedding objects, each containing an embedding vector and metadata.
Old:
const response = await ai.embed({embedder, content, options}); // returns number[]
New:
const response = await ai.embed({embedder, content, options}); // returns Embedding[]
const firstEmbeddingVector = response[0].embedding; // is number[]