Prompt Optimization Curves on BBH Tasks

Figures 23 and 24 illustrate prompt optimization curves for 21 BBH tasks, showcasing consistent upward trends when utilizing the text-bison scorer and PaLM 2-L-IT optimizer. This indicates effective optimization across multiple tasks.


This content originally appeared on HackerNoon and was authored by Writings, Papers and Blogs on Text Models

:::info Authors:

(1) Chengrun Yang, Google DeepMind and Equal contribution;

(2) Xuezhi Wang, Google DeepMind;

(3) Yifeng Lu, Google DeepMind;

(4) Hanxiao Liu, Google DeepMind;

(5) Quoc V. Le, Google DeepMind;

(6) Denny Zhou, Google DeepMind;

(7) Xinyun Chen, Google DeepMind and Equal contribution.

:::

Abstract and 1. Introduction

2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms

2.2 Meta-Prompt Design

3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression

3.2 Traveling Salesman Problem (TSP)

4 Application: Prompt Optimization and 4.1 Problem Setup

4.2 Meta-Prompt Design

5 Prompt Optimization Experiments and 5.1 Evaluation Setup

5.2 Main Results

5.3 Ablation Studies

5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt

6 Related Work

7 Conclusion, Acknowledgments and References

A Some Failure Cases

B Prompting Formats for Scorer Llm

C Meta-Prompts and C.1 Meta-Prompt for Math Optimization

C.2 Meta-Prompt for Prompt Optimization

D Prompt Optimization Curves on the Remaining Bbh Tasks

E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions

D PROMPT OPTIMIZATION CURVES ON THE REMAINING BBH TASKS

Figure 23: Prompt optimization on 21 BBH tasks (except ruin_names and temporal_sequences already shown in Figure 6) with the text-bison scorer and the PaLM 2-L-IT optimizer, Part I. Most curves have upward trends.

\ Figure 24: Prompt optimization on 21 BBH tasks (except ruin_names and temporal_sequences in Figure 6) with the text-bison scorer and the PaLM 2-L-IT optimizer, Part II. All curves have upward trends.

\

:::info This paper is available on arxiv under CC0 1.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by Writings, Papers and Blogs on Text Models


Print Share Comment Cite Upload Translate Updates
APA

Writings, Papers and Blogs on Text Models | Sciencx (2024-09-25T12:00:30+00:00) Prompt Optimization Curves on BBH Tasks. Retrieved from https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/

MLA
" » Prompt Optimization Curves on BBH Tasks." Writings, Papers and Blogs on Text Models | Sciencx - Wednesday September 25, 2024, https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/
HARVARD
Writings, Papers and Blogs on Text Models | Sciencx Wednesday September 25, 2024 » Prompt Optimization Curves on BBH Tasks., viewed ,<https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/>
VANCOUVER
Writings, Papers and Blogs on Text Models | Sciencx - » Prompt Optimization Curves on BBH Tasks. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/
CHICAGO
" » Prompt Optimization Curves on BBH Tasks." Writings, Papers and Blogs on Text Models | Sciencx - Accessed . https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/
IEEE
" » Prompt Optimization Curves on BBH Tasks." Writings, Papers and Blogs on Text Models | Sciencx [Online]. Available: https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/. [Accessed: ]
rf:citation
» Prompt Optimization Curves on BBH Tasks | Writings, Papers and Blogs on Text Models | Sciencx | https://www.scien.cx/2024/09/25/prompt-optimization-curves-on-bbh-tasks/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts