This content originally appeared on HackerNoon and was authored by Cosmological thinking: time, space and universal causation
:::info Authors:
(1) Zhan Ling, UC San Diego and equal contribution;
(2) Yunhao Fang, UC San Diego and equal contribution;
(3) Xuanlin Li, UC San Diego;
(4) Zhiao Huang, UC San Diego;
(5) Mingu Lee, Qualcomm AI Research and Qualcomm AI Research
(6) Roland Memisevic, Qualcomm AI Research;
(7) Hao Su, UC San Diego.
:::
Table of Links
Motivation and Problem Formulation
Deductively Verifiable Chain-of-Thought Reasoning
Conclusion, Acknowledgements and References
\ A Deductive Verification with Vicuna Models
C More Details on Answer Extraction
E More Deductive Verification Examples
C More Details on Answer Extraction
In this section, we describe our process to extract the final answer from language models’ responses. The process begins by selecting the last three non-empty lines. Then, these lines are processed through the following pipeline:
\
Firstly, we use a list of regular expressions to identify "No-Answer" patterns within the text, such as "we cannot answer (this|the) question". This process helps us ascertain whether the model can provide a conclusive answer. If any such patterns appear in the text, we mark "No answer!" as the final answer. However, if we don’t detect these patterns, we proceed to the next steps for extracting the final answer.
\
Secondly, if any "Answer-Split" patterns are found in the text, we divide the text into several blocks using the identified pattern. The last block of text is then utilized for extracting the answer.
\
Lastly, we use regular expressions, as outlined in Tab. 11, to scan the remaining text for possible final answers. If multiple matches are found for the pattern, we select the first match as the final answer. If no pattern matches are found in the remaining text, we default the final response to "No answer!".
\ “No-Answer” Patterns: "we cannot provide an answer to this question with (this|the) given information", "we cannot answer (this|the) question", "we cannot determine", "we can’t determine", "we do not have enough information to answer (this|the) question", "we do not have enough information to provide a definitive answer to (this|the) question", "the answer(.*?)is unknown", "answer is not listed among the answer choices".
\ “Answer-Split” Patterns: "answer is", "final answer:", "answer to the question is", "answer to this question is", "concatenated letters are", "concatenate the letters -", "The answer of ".
\
\
:::info This paper is available on arxiv under CC BY 4.0 DEED license.
:::
\
This content originally appeared on HackerNoon and was authored by Cosmological thinking: time, space and universal causation
Cosmological thinking: time, space and universal causation | Sciencx (2024-09-08T13:59:42+00:00) Deductive Verification of Chain-of-Thought Reasoning: More Details on Answer Extraction. Retrieved from https://www.scien.cx/2024/09/08/deductive-verification-of-chain-of-thought-reasoning-more-details-on-answer-extraction/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.