LangChain No using OpenAI API
(1) QA를 위한 Document 불러오기
# Load and process the text files
# loader = TextLoader("./data/texts")
loader = DirectoryLoader('./pdf/', glob="./*.pdf", loader_cls=PyPDFLoader)
documents = loader.load()
# Document 분절
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
texts = text_splitter.split_documents(documents)
(2) Embedding
# HuggingFaceInstructEmbedding
from langchain.embeddings import HuggingFaceInstructEmbeddings
instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl",
model_kwargs={"device": "cpu"}) # device "cuda"
OpenAIEmbedding을 사용하지 않고 HuggingFace의 Embedding 사용
(3) FastChat Model Import
import torch
import transformers
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained('lmsys/fastchat-t5-3b-v1.0')
model = AutoModelForSeq2SeqLM.from_pretrained('lmsys/fastchat-t5-3b-v1.0')
https://github.com/lm-sys/FastChat/blob/main/LICENSE
GitHub - lm-sys/FastChat: An open platform for training, serving, and evaluating large language models. Release repo for Vicuna
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and FastChat-T5. - GitHub - lm-sys/FastChat: An open platform for training, serving, and evalua...
github.com
fast-chat 상업적으로 이용이 가능
(4) pipeline task config
from transformers import pipeline
from langchain.llms import HuggingFacePipeline
import torch
pipe = pipeline(
"text2text-generation",
model=model,
tokenizer=tokenizer,
max_length=256
)
local_llm = HuggingFacePipeline(pipeline=pipe)
(5) RetrievalQA llm config
# create the chain to answer questions
qa_chain = RetrievalQA.from_chain_type(
llm=local_llm,
chain_type="stuff",
retriever=retriever,
#return_source_docummets=True,
)
(6) Test
query = "What is BERT language model?"
llm_response = qa_chain(query)
process_llm_response(llm_response)
<pad> BERT (Boolean Recurrent Entity Recognition) is a language model that is used to train and evaluate
natural language processing (NLP) systems. It is a probabilistic model that uses a combination of a
probabilistic representation of the input and a generative model to generate a sequence of recurrent units
(recurrent units). The BERT model is a subset of the generative model and is used to train NLP systems that
are trained to recognize and generate human language.
'Natural Language Processing' 카테고리의 다른 글
[LangChain] Sentence-Transformer (0) | 2023.06.01 |
---|---|
[OpenAI API] OpenAI Token (0) | 2023.05.30 |
[Mac] Transformer model downloaded path (0) | 2023.05.28 |
[LangChain] Retrieval PDF (0) | 2023.05.26 |
[LangChain] Building Custom Tool (0) | 2023.05.24 |