Skip to content

Conversation

@whatchang
Copy link

First, thank you for releasing your awesome paper and code.

I found one problem when I ran your Quick Start code that is written in README.
(That is below)

import argparse from tot.methods.bfs import solve from tot.tasks.game24 import Game24Task args = argparse.Namespace(backend='gpt-4', temperature=0.7, task='game24', naive_run=False, prompt_sample=None, method_generate='propose', method_evaluate='value', method_select='greedy', n_generate_sample=1, n_evaluate_sample=3, n_select_sample=5) task = Game24Task() ys, infos = solve(args, task, 900) print(ys[0]) 

The problematic line of code I found is shown below.
(Thas is in src->tot->tasks->game24.py line 91)

sum(value * value_names.count(name) for name, value in value_map.items()) 

In the above code, the problem was caused by the value_names being in the list type rather than a string.

As a result, we can see that the first string of value_names does not count even though it has 'sure' in it, as shown in the WATCH part of the figure below.

오류 n이 3일때

This also happens when n_evaluate_sample=1.

오류 n이 1일때

So, I modified the code to look like this.

sum(value * target_value.count(name) for name, value in value_map.items() for target_value in value_names) 

Thank you for reading and have a great day!

@ysymyth
Copy link
Member

ysymyth commented Jan 16, 2025

thanks! i think the problem is, what if target_value is a string like "sure sure likely impossible" or "i am sure i do not know"? exact match is more conservative but maybe less risky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants