【LLM、ChatGPT関連】Microsoft社のフレームワーク「guidance」を使ってみた
はじめに
ChatGPTのようなLLM(大規模言語モデル)の機能を拡張したり、便利に使うフレームワークとしてLangChainがメジャーですが、Microsoft社がgithubで提供している「guidance」というフレームワークを見つけました。
LangChainは機能豊富で、LLMに関連することは何でもできそうなほどいろいろ機能が揃っています。
ただ、機能が豊富過ぎて初心者には「これをどう役立てるのか?」を考えるのがちょっと難しいなーと思っていました。
使ってみて面白いし、面白いものは作れるのですが、
「プロダクトとして役に立つものを提供したい場合、どう活用しようか?」
と考えたときに少し悩んでしまいます。
そこで「LangChain」に代わるフレームワークが何かないかと探していてみつけたのがこの「guidance」です。
guidanceとは?
githubの説明は以下のとおりです。
この説明を読んでもさっぱりわかりませんでした。 笑
以下の特徴を見るともう少しわかりやすいような気がします。
1つ目にある「Handlebarsテンプレートに基づく」というのがポイントだと想像しました。
{{#user~}}
~ ~
{{~/user}}
という記述ができたり、
{{role}}: {{first_question}}{{/if}}
のようにロールの定義とかもシンプルでわかりやすいのが特徴かもしれません。
あと、確実にJSON形式での出力を期待したいときに使える機能もあるようなので、このあたりもメリットの1つかなと思っています。
LLM単体では、プロンプトで頑張って「JSONで回答してください」と指示しても、なかなか期待どおりのJSONになってくれないのでそういった使い方はこのguidanceの特徴かなと思います。
以下いくつかサンプルを試してみました。
インストール
インストールは簡単で以下のとおりです。
# pip install guidance
サンプル実行
Agent
2人のエージェントが相互に会話することをシミュレートするコードです。
import guidance
import re
guidance.llm = guidance.llms.OpenAI("gpt-4")
role_simulator = guidance('''
{{#system~}}
You are a helpful assistant
{{~/system}}
{{#user~}}
You will answer the user as {{role}} in the following conversation. At every step, I will provide you with the user input, as well as a comment reminding you of your instructions. Never talk about the fact that you are an AI, even if the user asks you. Always answer as {{role}}.
{{#if first_question}}You can also start the conversation.{{/if}}
{{~/user}}
{{~! The assistant either starts the conversation or not, depending on if this is the first or second agent }}
{{#assistant~}}
Ok, I will follow these instructions.
{{#if first_question}}Let me start the conversation now:
{{role}}: {{first_question}}{{/if}}
{{~/assistant}}
{{~! Then the conversation unrolls }}
{{~#geneach 'conversation' stop=False}}
{{#user~}}
User: {{set 'this.input' (await 'input')}}
Comment: Remember, answer as a {{role}}. Start your utterance with {{role}}:
{{~/user}}
{{#assistant~}}
{{gen 'this.response' temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}''')
republican = role_simulator(role='Republican', await_missing=True)
democrat = role_simulator(role='Democrat', await_missing=True)
first_question = '''What do you think is the best way to stop inflation?'''
republican = republican(input=first_question, first_question=None)
democrat = democrat(input=republican["conversation"][-2]["response"].strip('Republican: '), first_question=first_question)
for i in range(2):
republican = republican(input=democrat["conversation"][-2]["response"].replace('Democrat: ', ''))
democrat = democrat(input=republican["conversation"][-2]["response"].replace('Republican: ', ''))
print('Democrat: ' + first_question)
for x in democrat['conversation'][:-1]:
print('Republican:', x['input'])
print()
print(x['response'])
実行結果
Democrat: What do you think is the best way to stop inflation?
Republican: The best way to stop inflation is by implementing sound fiscal policies, such as reducing government spending, lowering taxes, and promoting economic growth. Additionally, the Federal Reserve should focus on maintaining a stable monetary policy to control inflation.
Democrat: While I agree that sound fiscal policies are important, as a Democrat, I believe that the government should invest in social programs, education, and infrastructure to promote economic growth and reduce income inequality. Additionally, the Federal Reserve should maintain a balance between controlling inflation and promoting full employment.
Republican: While investing in social programs, education, and infrastructure is important, it's crucial to prioritize fiscal responsibility and not overburden taxpayers. We believe that lowering taxes and reducing regulations can stimulate economic growth, which in turn can create more job opportunities and reduce income inequality. As for the Federal Reserve, maintaining a stable monetary policy to control inflation should remain its primary focus, as it ultimately benefits the overall economy.
Democrat: I understand your perspective, but as a Democrat, I believe that a progressive tax system and responsible government spending can help address income inequality and provide essential services to those in need. While reducing regulations can stimulate economic growth, it's important to ensure that we protect workers' rights, the environment, and consumer safety. As for the Federal Reserve, balancing inflation control with promoting full employment can lead to a more inclusive and stable economy for everyone.
Republican: We appreciate your perspective, but as Republicans, we believe that a progressive tax system can sometimes discourage investment and hinder economic growth. By lowering taxes and reducing regulations, we can create a more business-friendly environment that encourages job creation and innovation. While we agree that workers' rights, the environment, and consumer safety are important, we believe that a balanced approach to regulation is necessary to avoid stifling economic growth. As for the Federal Reserve, we maintain that controlling inflation should be its primary focus, as it ensures a stable and predictable economic environment that benefits everyone in the long run.
Democrat: I appreciate your viewpoint, but as a Democrat, I believe that a progressive tax system can help fund essential programs and services that benefit society as a whole. We can strike a balance between encouraging investment and ensuring that everyone pays their fair share. While we understand the importance of a business-friendly environment, we must also prioritize social and environmental responsibilities. A balanced approach to regulation can protect workers, the environment, and consumers without hindering economic growth. Regarding the Federal Reserve, we believe that a dual mandate of controlling inflation and promoting full employment can lead to a more inclusive and prosperous economy for all.
訳
Chat dialog
コード
import guidance
# connect to a chat model like GPT-4 or Vicuna
gpt4 = guidance.llms.OpenAI("gpt-4")
# vicuna = guidance.llms.transformers.Vicuna("your_path/vicuna_13B", device_map="auto")
experts = guidance('''
{{#system~}}
You are a helpful and terse assistant.
{{~/system}}
{{#user~}}
I want a response to the following question:
{{query}}
Name 3 world-class experts (past or present) who would be great at answering this?
Don't answer the question yet.
{{~/user}}
{{#assistant~}}
{{gen 'expert_names' temperature=0 max_tokens=300}}
{{~/assistant}}
{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
{{~/user}}
{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}
''', llm=gpt4)
res = experts(query='どうすれば生産性を高めることができますか?日本語でお願いします?')
print(res)
実行結果
<|im_start|>system
You are a helpful and terse assistant.<|im_end|>
<|im_start|>user
I want a response to the following question:
どうすれば生産性を高めることができますか?日本語でお願いします?
Name 3 world-class experts (past or present) who would be great at answering this?
Don't answer the question yet.<|im_end|>
<|im_start|>assistant
1. 松下幸之助
2. 稲盛和夫
3. 堀場雅夫<|im_end|>
<|im_start|>user
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.<|im_end|>
<|im_start|>assistant
生産性を高めるためには、以下の3つのポイントが重要です。
1. 目標設定と計画: 明確な目標を設定し、計画を立てることで、効率的に作業を進めることができます。目標は具体的で達成可能な ものにし、計画は柔軟性を持たせることが大切です。
2. コミュニケーションとチームワーク: チーム内での円滑なコミュニケーションが生産性向上に繋がります。意思疎通を図り、互い の役割を理解し合うことで、効率的な協力が可能になります。
3. 継続的な改善と学習: 常に改善の余地を見つけ、学び続けることで、生産性を高めることができます。失敗を恐れず、新しい方法 や技術を取り入れることが重要です。<|im_end|>
Rich output structure example
コード
import guidance
# set the default language model used to execute guidance programs
guidance.llm = guidance.llms.OpenAI("text-davinci-003")
# define the few shot examples
examples = [
{'input': 'I wrote about shakespeare',
'entities': [{'entity': 'I', 'time': 'present'}, {'entity': 'Shakespeare', 'time': '16th century'}],
'reasoning': 'I can write about Shakespeare because he lived in the past with respect to me.',
'answer': 'No'},
{'input': 'Shakespeare wrote about me',
'entities': [{'entity': 'Shakespeare', 'time': '16th century'}, {'entity': 'I', 'time': 'present'}],
'reasoning': 'Shakespeare cannot have written about me, because he died before I was born',
'answer': 'Yes'}
]
# define the guidance program
structure_program = guidance(
'''Given a sentence tell me whether it contains an anachronism (i.e. whether it could have happened or not based on the time periods associated with the entities).
----
{{~! display the few-shot examples ~}}
{{~#each examples}}
Sentence: {{this.input}}
Entities and dates:{{#each this.entities}}
{{this.entity}}: {{this.time}}{{/each}}
Reasoning: {{this.reasoning}}
Anachronism: {{this.answer}}
---
{{~/each}}
{{~! place the real question at the end }}
Sentence: {{input}}
Entities and dates:
{{gen "entities"}}
Reasoning:{{gen "reasoning"}}
Anachronism:{{#select "answer"}} Yes{{or}} No{{/select}}''')
# execute the program
out = structure_program(
examples=examples,
input='The T-rex bit my dog'
)
print(out)
実行結果
Given a sentence tell me whether it contains an anachronism (i.e. whether it could have happened or not based on the time periods associated with the entities).
----{{~! display the few-shot examples ~}}
Sentence: I wrote about shakespeare
Entities and dates:
I: present
Shakespeare: 16th century
Reasoning: I can write about Shakespeare because he lived in the past with respect to me.
Anachronism: No
---
Sentence: Shakespeare wrote about me
Entities and dates:
Shakespeare: 16th century
I: present
Reasoning: Shakespeare cannot have written about me, because he died before I was born
Anachronism: Yes
---
{{~! place the real question at the end }}
Sentence: The T-rex bit my dog
Entities and dates:
T-rex: Late Cretaceous period (66-145 million years ago)
My dog: Present
Reasoning: A T-rex cannot have bitten my dog, because they lived millions of years apart.
Anachronism: Yes
Reasoning:
Anachronism: Yes
訳
まとめ
guidanceの説明を読んでも、何がうれしいのかさっぱりわからなかったですが、サンプルを使ってみるとLLMからの応答に対してその後の動作を比較的的確にコントロールできそうかなという印象でした。
例えば、LangChainでもAgent機能はありますが、自分で細部までコントールしたい場合は何となくguidanceのほうが使い勝手は良さそうです。
(LangChainを使いこなしてないだけの可能性はありますので、あくまでも想像ですが)
とはいえ、サンプルを少し触ってみた程度なので、もう少し使い物になるフレームワークかどうかは探っていきたいと思います。
この記事が気に入ったらサポートをしてみませんか?