Skip to main content

LangServe

LangServe is a Python framework that helps developers deploy LangChain runnables and chains as REST APIs.

If you have a deployed LangServe route, you can use the RemoteRunnable class to interact with it as if it were a local chain. This allows you to more easily call hosted LangServe instances from JavaScript environments (like in the browser on the frontend).

You'll need to install or package LangChain into your frontend:

npm install langchain

Usage​

Then, you can use any of the supported LCEL interface methods. Here's an example of how this looks:

import { RemoteRunnable } from "langchain/runnables/remote";

const remoteChain = new RemoteRunnable({
  url: "https://your_hostname.com/path",
});

const result = await remoteChain.invoke({
  param1: "param1",
  param2: "param2",
});

console.log(result);

const stream = await remoteChain.stream({
  param1: "param1",
  param2: "param2",
});

for await (const chunk of stream) {
  console.log(chunk);
}

API Reference:

streamLog is a lower level method for streaming chain intermediate steps as partial JSONPatch chunks. This method allows for a few extra options as well to only include or exclude certain named steps.

@langchain/core also provides an applyPatch utility for aggregating these chunks into a full output:

import { RemoteRunnable } from "langchain/runnables/remote";
import { applyPatch } from "@langchain/core/utils/json_patch";

const remoteChain = new RemoteRunnable({
  url: "https://your_hostname.com/path",
});

const logStream = await remoteChain.streamLog(
  {
    param1: "param1",
    param2: "param2",
  },
  // LangChain runnable config properties
  {
    configurable: {
      llm: "some_property",
    },
    metadata: {
      conversation_id: "other_metadata",
    },
  },
  // Optional additional streamLog properties for filtering outputs
  {
    includeNames: [],
    includeTags: [],
    includeTypes: [],
    excludeNames: [],
    excludeTags: [],
    excludeTypes: [],
  }
);

let streamedResponse: Record<string, any> = {};

for await (const chunk of logStream) {
  console.log(chunk);
  streamedResponse = applyPatch(streamedResponse, chunk.ops).newDocument;
}

console.log(streamedResponse);

API Reference:

Configuration​

You can also pass in options for headers and timeouts into the constructor. These will be used when making incoming requests:

import { RemoteRunnable } from "langchain/runnables/remote";

const remoteChain = new RemoteRunnable({
  url: "https://your_hostname.com/path",
  options: {
    timeout: 10000,
    headers: {
      Authorization: "Bearer YOUR_TOKEN",
    },
  },
});

const result = await remoteChain.invoke({
  param1: "param1",
  param2: "param2",
});

console.log(result);

API Reference: