ModuleNotFoundError: No module named 'yaml' when trying to fit sagemaker tensorflow estimator locally

Issue

I am trying to follow this tutorial for connecting aws to a jupyter notebook for local development (I am running jupyter inside of vscode which I don’t think matters but just noting it in case).

I have this ipynb file from the tutorial up and have gotten it to run successfully all the way until the cell where you try to fit the tensorflow estimator on the data:

code looks like this:

 # Train! This will pull (once) the SageMaker CPU/GPU container for TensorFlow to your local machine.
# Make sure that Docker is running and that docker-compose is installed

tf_estimator.fit({'training': training_input_path, 'validation': validation_input_path})

I have docker installed but I don’t really know what I’m doing with it since I’ve never used it before. I can’t tell if this is a docker issue or a sagemaker issue but this is the error that is thrown when I try to run that cell:

Failed to import yaml. Local mode features will be impaired or broken. Please run "pip install 'sagemaker[local]'" to install all required dependencies.
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
/var/folders/cf/ft88j_856fv5rk3whgs12d9w0000gq/T/ipykernel_97461/84733352.py in <module>
      2 # Make sure that Docker is running and that docker-compose is installed
      3 
----> 4 tf_estimator.fit({'training': training_input_path, 'validation': validation_input_path})

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/workflow/pipeline_context.py in wrapper(*args, **kwargs)
    207             return self_instance.sagemaker_session.context
    208 
--> 209         return run_func(*args, **kwargs)
    210 
    211     return wrapper

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)
    976         self._prepare_for_training(job_name=job_name)
    977 
--> 978         self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config)
    979         self.jobs.append(self.latest_training_job)
    980         if wait:

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/estimator.py in start_new(cls, estimator, inputs, experiment_config)
   1806         train_args = cls._get_train_args(estimator, inputs, experiment_config)
   1807 
-> 1808         estimator.sagemaker_session.train(**train_args)
   1809 
   1810         return cls(estimator.sagemaker_session, estimator._current_job_name)

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/session.py in train(self, input_mode, input_config, role, job_name, output_config, resource_config, vpc_config, hyperparameters, stop_condition, tags, metric_definitions, enable_network_isolation, image_uri, algorithm_arn, encrypt_inter_container_traffic, use_spot_instances, checkpoint_s3_uri, checkpoint_local_path, experiment_config, debugger_rule_configs, debugger_hook_config, tensorboard_output_config, enable_sagemaker_metrics, profiler_rule_configs, profiler_config, environment, retry_strategy)
    592             self.sagemaker_client.create_training_job(**request)
    593 
--> 594         self._intercept_create_request(train_request, submit, self.train.__name__)
    595 
    596     def _get_train_request(  # noqa: C901

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/session.py in _intercept_create_request(self, request, create, func_name)
   4201             func_name (str): the name of the function needed intercepting
   4202         """
-> 4203         return create(request)
   4204 
   4205 

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/session.py in submit(request)
    590             LOGGER.info("Creating training-job with name: %s", job_name)
    591             LOGGER.debug("train request: %s", json.dumps(request, indent=4))
--> 592             self.sagemaker_client.create_training_job(**request)
    593 
    594         self._intercept_create_request(train_request, submit, self.train.__name__)

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/local/local_session.py in create_training_job(self, TrainingJobName, AlgorithmSpecification, OutputDataConfig, ResourceConfig, InputDataConfig, Environment, **kwargs)
    190         logger.info("Starting training job")
    191         training_job.start(
--> 192             InputDataConfig, OutputDataConfig, hyperparameters, Environment, TrainingJobName
    193         )
    194 

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/local/entities.py in start(self, input_data_config, output_data_config, hyperparameters, environment, job_name)
    235 
    236         self.model_artifacts = self.container.train(
--> 237             input_data_config, output_data_config, hyperparameters, environment, job_name
    238         )
    239         self.end_time = datetime.datetime.now()
~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/local/image.py in train(self, input_data_config, output_data_config, hyperparameters, environment, job_name)
    234 
    235         compose_data = self._generate_compose_file(
--> 236             "train", additional_volumes=volumes, additional_env_vars=training_env_vars
    237         )
    238         compose_command = self._compose()

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/local/image.py in _generate_compose_file(self, command, additional_volumes, additional_env_vars)
    687         except ImportError as e:
    688             logger.error(sagemaker.utils._module_import_error("yaml", "Local mode", "local"))
--> 689             raise e
    690 
    691         yaml_content = yaml.dump(content, default_flow_style=False)

~/opt/anaconda3/envs/localsm/lib/python3.7/site-packages/sagemaker/local/image.py in _generate_compose_file(self, command, additional_volumes, additional_env_vars)
    684 
    685         try:
--> 686             import yaml
    687         except ImportError as e:
    688             logger.error(sagemaker.utils._module_import_error("yaml", "Local mode", "local"))

ModuleNotFoundError: No module named 'yaml'

Wondering if anyone else has had this happen or if someone can point me in the right direction to try and figure out what the issue is and how to solve it. Thanks!

Solution

You are just missing the pyyaml module

Install it by:

pip install pyyaml

Answered By – Toby Cm

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published