[MTurk] You are not currently qualified to work on this task

[molingbo]

Hi,

I am using Mephisto to do some local tests for a crowdsourcing task. I set the provider of Mephisto as 'mock'. When I launched a task and tried to get access to it on the browser, it just showed an alert message like: "You are not currently qualified to work on this task". I didn't set any qualifications and block list. Not sure what happened there. It'd be great if you could share some ideas or relevant information with me. Thanks!

[JackUrb]

Hi @molingbo - can you share your hydra config? And perhaps your logs when using mephisto.log_level=debug? I'm not sure how to trace this down with the current information, though it's certainly an unexpected occurrence if you're not using any qualifications!

This specifically comes from being dropped with WorkerFailureReasons.NOT_QUALIFIED, which occurs in a few places in the WorkerPool. Either the worker_is_qualified check fails (running against any qualifications set for the task), or the associated worker fails the onboarding check (has been disqualified in onboarding).

Perhaps you have onboarding set up? You can try accessing as another worker by changing the worker_id parameter in the URL.

[molingbo]

@JackUrb Thanks for the quick response! I am skipping onboarding step actually and here's the hydra config:

mephisto:
blueprint:
_blueprint_type: model_chat_blueprint
block_qualification: None
onboarding_qualification: None
_group: ModelChatBlueprint
world_file: /home/ParlAI/parlai/crowdsourcing/tasks/model_chat/worlds.py
preview_source: ???
task_description_file: ${mephisto.blueprint.task_config_path}/task_description.html
custom_source_bundle: ???
custom_source_dir: /home/ParlAI/parlai/crowdsourcing/tasks/model_chat/frontend
extra_source_dir: ???
context_csv: ???
context_jsonl: ???
num_conversations: 6
num_turns: 6
random_seed: 42
annotation_question: Please indicate all the necessary changes to make the sub-questions
accurately reflect the meaning of the target question.
model_opt_path: ${task_dir}/task_config/model_opts.yaml
task_model_parallel: true
max_resp_time: 900
chat_data_folder: ${task_dir}/model_chat/
check_acceptability: false
context_seed: ???
task_config_path: /home/ParlAI/parlai/crowdsourcing/tasks/model_chat/task_config
left_pane_text_path: ${mephisto.blueprint.task_config_path}/left_pane_text.html
annotations_config_path: ${task_dir}/task_config/annotations_config.json
final_rating_question: Please rate your partner on a scale of 1-5.
max_concurrent_responses: 1
override_opt: {}
conversation_start_mode: hi
include_persona: false
conversations_needed_string: "blender_90M:10"
max_onboard_time: 500
onboard_task_data_path: ${mephisto.blueprint.task_config_path}/onboard_task_data.json
provider:
_provider_type: mock
requester_name: MOCK_REQUESTER
architect:
_architect_type: local
server_type: node
server_source_path: ???
hostname: localhost
port: '3000'
task:
task_name: model_chat
task_title: model chat
task_description: This is a ParlAI model chat task.
task_reward: 0.05
task_tags: chat,conversation,dialog,partner
assignment_duration_in_seconds: 6000
allowed_concurrent: 3
maximum_units_per_worker: 3
max_num_concurrent_units: 0
submission_timeout: 600
database:
_database_type: singleton
log_level: info
task_dir: /home/ParlAI/parlai/crowdsourcing/tasks/model_chat
num_tasks: 5
current_time: 1652308365
mturk:
worker_blocklist_paths: null
monitoring_log_rate: 30`

[JackUrb]

Does the issue persist if you use a different worker_id in the mock?

[molingbo]

Yeah, I tried that and the issue was still there. But I tried to directly comment out the onboarding_qualification argument in the hydra_configs/conf/example.yaml. It seems it works now, but not sure whether this is another normal way to skip qualification or if there's any side effect for doing so.