Skip to main content

在Intel CPU上使用IPEX-LLM的本地BGE嵌入

IPEX-LLM 是一个用于在Intel CPU和GPU(例如,带有iGPU的本地PC、离散GPU如Arc、Flex和Max)上运行大型语言模型的PyTorch库,具有非常低的延迟。

本示例介绍了如何使用LangChain在Intel CPU上进行嵌入任务,并利用ipex-llm优化。这在RAG、文档问答等应用中将非常有帮助。

设置

%pip install -qU langchain langchain-community

安装 IPEX-LLM 以在 Intel CPU 上进行优化,以及 sentence-transformers

%pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu
%pip install sentence-transformers

注意

对于 Windows 用户,安装 ipex-llm 时不需要 --extra-index-url https://download.pytorch.org/whl/cpu

基本用法

<!--IMPORTS:[{"imported": "IpexLLMBgeEmbeddings", "source": "langchain_community.embeddings", "docs": "https://python.langchain.com/api_reference/community/embeddings/langchain_community.embeddings.ipex_llm.IpexLLMBgeEmbeddings.html", "title": "Local BGE Embeddings with IPEX-LLM on Intel CPU"}]-->
from langchain_community.embeddings import IpexLLMBgeEmbeddings

embedding_model = IpexLLMBgeEmbeddings(
model_name="BAAI/bge-large-en-v1.5",
model_kwargs={},
encode_kwargs={"normalize_embeddings": True},
)

API 参考

sentence = "IPEX-LLM is a PyTorch library for running LLM on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) with very low latency."
query = "What is IPEX-LLM?"

text_embeddings = embedding_model.embed_documents([sentence, query])
print(f"text_embeddings[0][:10]: {text_embeddings[0][:10]}")
print(f"text_embeddings[1][:10]: {text_embeddings[1][:10]}")

query_embedding = embedding_model.embed_query(query)
print(f"query_embedding[:10]: {query_embedding[:10]}")

相关


Was this page helpful?


You can also leave detailed feedback on GitHub.

扫我,入群扫我,找书