Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provided Pre-trained ShapeNet Segmentation Model FAILS to Restore #232

Open
iskandergaba opened this issue May 22, 2020 · 2 comments
Open

Provided Pre-trained ShapeNet Segmentation Model FAILS to Restore #232

iskandergaba opened this issue May 22, 2020 · 2 comments

Comments

@iskandergaba
Copy link

Hello,

I tried to run the testing on the ShapeNet pre-trained segmentation model that you provided. I followed all the instructions and ran the scripts to download and prepare the data. I also compiled the custom FarthestPointSampling operator successfully.

Then I used this command below to run the testing from pointcnn_seg directory:

./test_shapenet.sh -g 0 -x shapenet_x8_2048_fps -l ../../models/seg/pointcnn_seg_shapenet_x8_2048_fps_xxxx/ckpts/iter-544500 -r 10

Needless to say, I put the checkpoint files in this directory (relative to pointcnn_seg):

../../models/seg/pointcnn_seg_shapenet_x8_2048_fps_xxxx/ckpts/

The model apparently fails to load, giving errors regarding FPS and stating that:

Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint.

I do not understand. Could it be a problem with the provided model? Or am I doing anything wrong?
I would appreciate any insight on this.

Thank you.

Full stack trace:

Traceback (most recent call last):
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1317, in _run_fn
    self._extend_graph()
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1352, in _extend_graph
    tf_session.ExtendSession(self._session)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'FarthestPointSample' used by {{node FarthestPointSample}}with these attrs: [npoint=768]
Registered devices: [CPU, XLA_CPU]
Registered kernels:
  device='GPU'

	 [[{{node FarthestPointSample}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 1276, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'FarthestPointSample' used by node FarthestPointSample (defined at <string>:148) with these attrs: [npoint=768]
Registered devices: [CPU, XLA_CPU]
Registered kernels:
  device='GPU'

	 [[node FarthestPointSample (defined at <string>:148) ]]

Caused by op 'FarthestPointSample', defined at:
  File "../test_shapenet_seg.py", line 156, in <module>
    main()
  File "../test_shapenet_seg.py", line 92, in main
    net = model.Net(points_sampled, features_sampled, is_training, setting)
  File "/home/igaba/miniconda2/envs/pointcnn/src/pointcnn_seg.py", line 11, in __init__
    PointCNN.__init__(self, points, features, is_training, setting)
  File "/home/igaba/miniconda2/envs/pointcnn/src/pointcnn.py", line 93, in __init__
    fps_indices = tf_sampling.farthest_point_sample(P, pts)
  File "/home/igaba/miniconda2/envs/pointcnn/src/sampling/tf_sampling.py", line 59, in farthest_point_sample
    return sampling_module.farthest_point_sample(inp, npoint)
  File "<string>", line 148, in farthest_point_sample
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): No OpKernel was registered to support Op 'FarthestPointSample' used by node FarthestPointSample (defined at <string>:148) with these attrs: [npoint=768]
Registered devices: [CPU, XLA_CPU]
Registered kernels:
  device='GPU'

	 [[node FarthestPointSample (defined at <string>:148) ]]


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../test_shapenet_seg.py", line 156, in <module>
    main()
  File "../test_shapenet_seg.py", line 103, in main
    saver.restore(sess, args.load_ckpt)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 1312, in restore
    err, "a mismatch between the current graph and the graph")
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

No OpKernel was registered to support Op 'FarthestPointSample' used by node FarthestPointSample (defined at <string>:148) with these attrs: [npoint=768]
Registered devices: [CPU, XLA_CPU]
Registered kernels:
  device='GPU'

	 [[node FarthestPointSample (defined at <string>:148) ]]

Caused by op 'FarthestPointSample', defined at:
  File "../test_shapenet_seg.py", line 156, in <module>
    main()
  File "../test_shapenet_seg.py", line 92, in main
    net = model.Net(points_sampled, features_sampled, is_training, setting)
  File "/home/igaba/miniconda2/envs/pointcnn/src/pointcnn_seg.py", line 11, in __init__
    PointCNN.__init__(self, points, features, is_training, setting)
  File "/home/igaba/miniconda2/envs/pointcnn/src/pointcnn.py", line 93, in __init__
    fps_indices = tf_sampling.farthest_point_sample(P, pts)
  File "/home/igaba/miniconda2/envs/pointcnn/src/sampling/tf_sampling.py", line 59, in farthest_point_sample
    return sampling_module.farthest_point_sample(inp, npoint)
  File "<string>", line 148, in farthest_point_sample
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/home/igaba/miniconda2/envs/pointcnn/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

No OpKernel was registered to support Op 'FarthestPointSample' used by node FarthestPointSample (defined at <string>:148) with these attrs: [npoint=768]
Registered devices: [CPU, XLA_CPU]
Registered kernels:
  device='GPU'

	 [[node FarthestPointSample (defined at <string>:148) ]]
@aniketpant83
Copy link

Hey! did you get an answer for the above?

@iskandergaba
Copy link
Author

iskandergaba commented Dec 2, 2020 •
edited

Hey! did you get an answer for the above?

Hi, nope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@iskandergaba @aniketpant83