deep learning - torch cannot load a pth file containing plotly figures - Stack Overflow

admin管理员组

文章数量:1391795

I am running a bunch of DNN experiments using pytorch, and saving an instance of my ExperimentClass as a .pth file. The ExperimentClass has instance variables including a torch model, a dictionary of multiple plotly figure objects, dataframes, and other metadata related to experimental parameters.

However, when I try to load the .pth file using torch.load(full_model_path, weights_only=False), I get a cryptic ValueError related to some figure type that I never even use, but it's definitely occurring during the unpacking of the plotly figure objects.

ValueError: Invalid property specified for object of type plotly.graph_objs.layout.template.Data: 'heatmapgl'

Here is the full traceback:

loading model from experiment_plots/vLAYER/dnn_all_experiments_results.pth
Traceback (most recent call last):
  File "/Users/name/Desktop/name/research/nn-layer-weight-experiments/create_additional_training_plots.py", line 163, in <module>
    create_final_epoch_layer_plots(experiment_versions=experiment_versions, dnn_names=dnn_names)
  File "/Users/name/Desktop/name/research/nn-layer-weight-experiments/create_additional_training_plots.py", line 34, in create_final_epoch_layer_plots
    dnn_experiments = torch.load(full_model_path, weights_only=False)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/torch/serialization.py", line 1471, in load
    return _load(
           ^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/torch/serialization.py", line 1964, in _load
    result = unpickler.load()
             ^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/_figure.py", line 641, in __init__
    super(Figure, self).__init__(data, layout, frames, skip_invalid, **kwargs)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 556, in __init__
    self._layout_obj = self._layout_validator.validate_coerce(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2504, in validate_coerce
    v = self.data_class(v, skip_invalid=skip_invalid, _validate=_validate)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/_layout.py", line 7124, in __init__
    self["template"] = _v
    ~~~~^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 5898, in __setitem__
    super(BaseLayoutHierarchyType, self).__setitem__(prop, value)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4852, in __setitem__
    self._set_compound_prop(prop, value)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 5263, in _set_compound_prop
    val = validator.validate_coerce(val, skip_invalid=self._skip_invalid)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2797, in validate_coerce
    return super(BaseTemplateValidator, self).validate_coerce(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2504, in validate_coerce
    v = self.data_class(v, skip_invalid=skip_invalid, _validate=_validate)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/layout/_template.py", line 327, in __init__
    self["data"] = _v
    ~~~~^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4852, in __setitem__
    self._set_compound_prop(prop, value)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 5263, in _set_compound_prop
    val = validator.validate_coerce(val, skip_invalid=self._skip_invalid)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2504, in validate_coerce
    v = self.data_class(v, skip_invalid=skip_invalid, _validate=_validate)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/layout/template/_data.py", line 1791, in __init__
    self._process_kwargs(**dict(arg, **kwargs))
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4378, in _process_kwargs
    raise err
ValueError: Invalid property specified for object of type plotly.graph_objs.layout.template.Data: 'heatmapgl'

What is also strange is that I ran a smaller version of my test experiments locally about a day ago, and all of the .pth files were able to be loaded without any errors. Has anyone else encountered this?

I am running a bunch of DNN experiments using pytorch, and saving an instance of my ExperimentClass as a .pth file. The ExperimentClass has instance variables including a torch model, a dictionary of multiple plotly figure objects, dataframes, and other metadata related to experimental parameters.

However, when I try to load the .pth file using torch.load(full_model_path, weights_only=False), I get a cryptic ValueError related to some figure type that I never even use, but it's definitely occurring during the unpacking of the plotly figure objects.

ValueError: Invalid property specified for object of type plotly.graph_objs.layout.template.Data: 'heatmapgl'

Here is the full traceback:

loading model from experiment_plots/vLAYER/dnn_all_experiments_results.pth
Traceback (most recent call last):
  File "/Users/name/Desktop/name/research/nn-layer-weight-experiments/create_additional_training_plots.py", line 163, in <module>
    create_final_epoch_layer_plots(experiment_versions=experiment_versions, dnn_names=dnn_names)
  File "/Users/name/Desktop/name/research/nn-layer-weight-experiments/create_additional_training_plots.py", line 34, in create_final_epoch_layer_plots
    dnn_experiments = torch.load(full_model_path, weights_only=False)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/torch/serialization.py", line 1471, in load
    return _load(
           ^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/torch/serialization.py", line 1964, in _load
    result = unpickler.load()
             ^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/_figure.py", line 641, in __init__
    super(Figure, self).__init__(data, layout, frames, skip_invalid, **kwargs)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 556, in __init__
    self._layout_obj = self._layout_validator.validate_coerce(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2504, in validate_coerce
    v = self.data_class(v, skip_invalid=skip_invalid, _validate=_validate)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/_layout.py", line 7124, in __init__
    self["template"] = _v
    ~~~~^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 5898, in __setitem__
    super(BaseLayoutHierarchyType, self).__setitem__(prop, value)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4852, in __setitem__
    self._set_compound_prop(prop, value)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 5263, in _set_compound_prop
    val = validator.validate_coerce(val, skip_invalid=self._skip_invalid)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2797, in validate_coerce
    return super(BaseTemplateValidator, self).validate_coerce(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2504, in validate_coerce
    v = self.data_class(v, skip_invalid=skip_invalid, _validate=_validate)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/layout/_template.py", line 327, in __init__
    self["data"] = _v
    ~~~~^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4852, in __setitem__
    self._set_compound_prop(prop, value)
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 5263, in _set_compound_prop
    val = validator.validate_coerce(val, skip_invalid=self._skip_invalid)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/_plotly_utils/basevalidators.py", line 2504, in validate_coerce
    v = self.data_class(v, skip_invalid=skip_invalid, _validate=_validate)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/graph_objs/layout/template/_data.py", line 1791, in __init__
    self._process_kwargs(**dict(arg, **kwargs))
  File "/Users/name/miniconda3/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4378, in _process_kwargs
    raise err
ValueError: Invalid property specified for object of type plotly.graph_objs.layout.template.Data: 'heatmapgl'

What is also strange is that I ran a smaller version of my test experiments locally about a day ago, and all of the .pth files were able to be loaded without any errors. Has anyone else encountered this?

• deep-learning
• pytorch
• plotly
• pth

Share Improve this question Follow edited Mar 14 at 0:16 Derek O asked Mar 12 at 5:08 Derek ODerek O 19.7k44 gold badges2929 silver badges4949 bronze badges 7

• Are you saving the model using the torch.save(*) method? What object are you saving, the instance of ExperimentClass or just the nn.Module object? – daqh Commented Mar 12 at 8:40
• 1 From which version of plotly did you save your results, with which version of plotly are you trying to load them? It seems like earlier versions of the Data class had a heatmapgl property (cf. here), while the newest versions no longer have it (cf. here). – simon Commented Mar 12 at 11:44
• 2 As a general advice, I would try to only save instances of built-in Python classes (numbers, strings, tuples, lists, dicts, etc.) and PyTorch tensors into PyTorch checkpoints and try to recreate other instances from values stored in these instances. In this way, in my experience, problems like yours (which is essentially the problem of unpickling an instance of a class whose definition has changed in-between pickling and unpickling) can be largely avoided. – simon Commented Mar 12 at 12:00
• 1 @simon yes that was it, thank you! I recently upgraded plotly to v6.0.0 (to help reproduce an error someone on SO had with plotly, funnily enough), but when I saved these instances of ExperimentClass objects, I had used Plotly v5.23.0. After downgrading Plotly to the earlier version, I am now able to successfully load the pth file – Derek O Commented Mar 12 at 20:50
• 1 @DerekO Thanks a lot for your replies – I am glad that you could solve your problem! I summarized the comments above into an answer. Feel free to adjust it as necessary. – simon Commented Mar 13 at 9:08

 |  Show 2 more comments

1 Answer 1

Sorted by: Reset to default 1

The following is a summary from the comments to the question:

In earlier versions of plotly, its Data class had a heatmapgl property (see e.g. an earlier version of Data's source code here), while in the most recent versions, this property has been removed (see the current version of Data's source code here).

The problem and its solution thus unfolded as follows:

• PyTorch experimental results, including instances of the Data class, were saved from a Python environment with an earlier version of plotly (v5.23.0 in this case), where the heatmapgl property was still present.
• When trying to load these results, the plotly version had already been updated (to v6.0.0 in this case) and the heatmapgl property was already gone, so unpickling/unserializing an instance of the Data class resulted in the reported error (ValueError: Invalid property specified for ….Data: 'heatmapgl').
• The problem was solved by downgrading plotly to its previously used version (v5.23.0).

My personal experience and advice for avoiding such a problem (which, as we have seen, essentially boils down to the problem of unpickling/unserializing an instance of a class whose definition has changed between pickling/serializing and unpickling/unserializing) is as follows: I would try to save only instances of built-in Python classes (numbers, strings, tuples, lists, dicts, etc.) as well as PyTorch tensors in PyTorch checkpoints, and try to recreate other instances from values stored in these instances.

本文标签： deep learningtorch cannot load a pth file containing plotly figuresStack Overflow