{"input":[{"id":"87ffaae3","content":"{\"state\": \"Where is my package? I ordered it last week and it still hasn't arrived.\", \"question\": {\"criteria\": {\"billing_question\": \"Asks about a charge, invoice or payment\", \"cancel_order\": \"Wants to cancel an order\", \"change_address\": \"Wants to change the delivery address\", \"report_damage\": \"Received an item that is broken or damaged\", \"track_order\": \"Wants to know where an order is or when it arrives\"}, \"instructions\": \"Which intent does the user's message express?\", \"type\": \"choice\"}}","internal":{"systemone":{"kind":"choice","state":"Where is my package? I ordered it last week and it still hasn't arrived.","instructions":"Which intent does the user's message express?","criteria":{"billing_question":"Asks about a charge, invoice or payment","cancel_order":"Wants to cancel an order","change_address":"Wants to change the delivery address","report_damage":"Received an item that is broken or damaged","track_order":"Wants to know where an order is or when it arrives"},"answer_prefix":"ANSWER: "}}}],"target":"track_order","id":0,"group_id":0,"metadata":{"source_id":"easy-intent-00","decision_gold":"track_order","decision_kind":"choice","source_group":null,"family":"intent","tier":"","language":"","continuous":false,"decision_labels":["billing_question","cancel_order","change_address","report_damage","track_order"],"fixed_classes":false}}
提示模板
未定义提示模板。
使用方法
使用 CLI
evalscopeeval\--modelYOUR_MODEL\--eval-typesystemone_api\--api-urlhttp://127.0.0.1:18131/v1\--api-keyEMPTY_TOKEN\--datasetsjevbench\--limit10# Remove this line for formal evaluation
使用 Python
fromevalscopeimportrun_taskfromevalscope.configimportTaskConfigtask_cfg=TaskConfig(model='YOUR_MODEL',eval_type='systemone_api',api_url='http://127.0.0.1:18131/v1',api_key='EMPTY_TOKEN',datasets=['jevbench'],limit=10,# Remove this line for formal evaluation)run_task(task_cfg=task_cfg)