> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-5e9cc07a-preview-update-1770920925-e15dbde.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# ChatFireworks integration
> Integrate with the ChatFireworks chat model using LangChain Python.
This doc helps you get started with Fireworks AI [chat models](/oss/python/langchain/models). For detailed documentation of all ChatFireworks features and configurations head to the [API reference](https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html).
Fireworks AI is an AI inference platform to run and customize models. For a list of all models served by Fireworks see the [Fireworks docs](https://fireworks.ai/models).
## Overview
### Integration details
| Class | Package | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/fireworks) | Downloads | Version |
| :----------------------------------------------------------------------------------- | :--------------------------------------------------------------------- | :----------: | :---------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------: |
| [`ChatFireworks`](https://docs.langchain.com/oss/python/integrations/chat/fireworks) | [`langchain-fireworks`](https://pypi.org/project/langchain-fireworks/) | beta | ✅ |  |  |
### Model features
| [Tool calling](/oss/python/langchain/tools) | [Structured output](/oss/python/langchain/structured-output) | [Image input](/oss/python/langchain/messages#multimodal) | Audio input | Video input | [Token-level streaming](/oss/python/langchain/streaming/) | Native async | [Token usage](/oss/python/langchain/models#token-usage) | [Logprobs](/oss/python/langchain/models#log-probabilities) |
| :-----------------------------------------: | :----------------------------------------------------------: | :------------------------------------------------------: | :---------: | :---------: | :-------------------------------------------------------: | :----------: | :-----------------------------------------------------: | :--------------------------------------------------------: |
| ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
## Setup
To access Fireworks models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.
### Credentials
Head to ([fireworks.ai/login](https://fireworks.ai/login) to sign up to Fireworks and generate an API key. Once you've done this set the FIREWORKS\_API\_KEY environment variable:
```python theme={null}
import getpass
import os
if "FIREWORKS_API_KEY" not in os.environ:
os.environ["FIREWORKS_API_KEY"] = getpass.getpass("Enter your Fireworks API key: ")
```
To enable automated tracing of your model calls, set your [LangSmith](https://docs.langchain.com/langsmith/home) API key:
```python theme={null}
os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
os.environ["LANGSMITH_TRACING"] = "true"
```
### Installation
The LangChain Fireworks integration lives in the `langchain-fireworks` package:
```python theme={null}
pip install -qU langchain-fireworks
```
## Instantiation
Now we can instantiate our model object and generate chat completions:
* TODO: Update model instantiation with relevant params.
```python theme={null}
from langchain_fireworks import ChatFireworks
llm = ChatFireworks(
model="accounts/fireworks/models/kimi-k2-instruct-0905", # Model library in: https://app.fireworks.ai/models
temperature=0,
max_tokens=None,
timeout=None,
max_retries=2,
# other params...
)
```
## Invocation
```python theme={null}
messages = [
(
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
),
("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg
```
```text theme={null}
AIMessage(content="J'adore la programmation.", additional_kwargs={}, response_metadata={'token_usage': {'prompt_tokens': 31, 'total_tokens': 41, 'completion_tokens': 10}, 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None, 'model_provider': 'fireworks', 'model_name': 'accounts/fireworks/models/kimi-k2-instruct-0905'}, id='lc_run--a2bdeca3-6394-4c80-97ad-2fc8db9f54bb-0', usage_metadata={'input_tokens': 31, 'output_tokens': 10, 'total_tokens': 41})
```
```python theme={null}
print(ai_msg.content)
```
```text theme={null}
J'adore la programmation.
```
## API reference
For detailed documentation of all ChatFireworks features and configurations head to the [API reference](https://reference.langchain.com/python/integrations/langchain_fireworks/)
To use the `langchain-fireworks` package, follow these installation steps:
```bash theme={null}
pip install langchain-fireworks
```
## Basic usage
### Setting up
1. Sign in to [Fireworks AI](http://fireworks.ai/) to obtain an API Key to access the models, and make sure it is set as the `FIREWORKS_API_KEY` environment variable.
Once you've signed in and obtained an API key, follow these steps to set the `FIREWORKS_API_KEY` environment variable:
* **Linux/macOS:** Open your terminal and execute the following command:
```bash theme={null}
export FIREWORKS_API_KEY='your_api_key'
```
**Note:** To make this environment variable persistent across terminal sessions, add the above line to your `~/.bashrc`, `~/.bash_profile`, or `~/.zshrc` file.
* **Windows:** For Command Prompt, use:
```cmd theme={null}
set FIREWORKS_API_KEY=your_api_key
```
2. Set up your model using a model id. If the model is not set, the default model is `fireworks-llama-v2-7b-chat`. See the full, most up-to-date model list on [fireworks.ai](https://app.fireworks.ai/models).
```python theme={null}
import getpass
import os
from langchain_fireworks import ChatFireworks
# Initialize a Fireworks model
llm = ChatFireworks(
model="accounts/fireworks/models/llama-v3p1-8b-instruct",
base_url="https://api.fireworks.ai/inference/v1/completions",
)
```
### Calling the model directly
You can call the model directly with string prompts to get completions.
```python theme={null}
# Single prompt
output = llm.invoke("Who's the best quarterback in the NFL?")
print(output)
```
```python theme={null}
# Calling multiple prompts
output = llm.generate(
[
"Who's the best cricket player in 2016?",
"Who's the best basketball player in the league?",
]
)
print(output.generations)
```
## Advanced usage
### Tool use: LangChain Agent + fireworks function calling model
Please checkout how to teach Fireworks function calling model to use a calculator [in this notebook](https://github.com/fw-ai/cookbook/blob/main/learn/function-calling/notebooks_langchain/fireworks_langchain_tool_usage.ipynb).
Fireworks focus on delivering the best experience for fast model inference as well as tool use. You can check out [our blog](https://fireworks.ai/blog/firefunction-v1-gpt-4-level-function-calling) for more details on how it compares to GPT-4, the punchline is that it is on par with GPT-4 in terms of function calling use cases, but it is way faster and much cheaper.
### RAG: LangChain agent + fireworks function calling model + MongoDB + nomic AI embeddings
Please check out the [cookbook here](https://github.com/fw-ai/cookbook/blob/main/integrations/MongoDB/project_rag_with_mongodb/mongodb_agent.ipynb) for an end to end flow
***
[Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/chat/fireworks.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
[Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.