Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scalable pipeline Segmentation Fault #1100

Open
xiaoxiae opened this issue Feb 1, 2024 · 11 comments
Open

Scalable pipeline Segmentation Fault #1100

xiaoxiae opened this issue Feb 1, 2024 · 11 comments

Comments

@xiaoxiae
Copy link

xiaoxiae commented Feb 1, 2024

Describe the bug
Running the steps outlined in the MvsScalablePipeline.py yields a segmentation fault.

To Reproduce
Running the following commands (after InterfaceCOLMAP):

  1. DensifyPointCloud scene.mvs --resolution-level 1 --iters 4 --cuda-device -1 --fusion-mode 1
  2. DensifyPointCloud scene.mvs --resolution-level 1 --iters 4 --cuda-device -1 --sub-scene-area 20000000
  3. DensifyPointCloud scene_0000.mvs --resolution-level 1 --iters 4 --cuda-device -1 --dense-config-file Densify.ini -v 3
  • the Densify.ini contents are Optimize = 0

Step (3) crashes with the following output:

20:48:49 [App     ] OpenMVS x64 v2.2.0
20:48:49 [App     ] Build date: Jan 25 2024, 10:11:25
20:48:49 [App     ] CPU: AMD Ryzen 7 1700 Eight-Core Processor           (16 cores)
20:48:49 [App     ] RAM: 62.74GB Physical Memory 32.00GB Virtual Memory
20:48:49 [App     ] OS: Linux 5.15.146-1-MANJARO (x86_64)
20:48:49 [App     ] Disk: 215.30GB (960.17GB) space
20:48:49 [App     ] SSE & AVX compatible CPU & OS detected
20:48:49 [App     ] Command line: DensifyPointCloud scene_0000.mvs --resolution-level 0 --iters 4 --cuda-device -1 --dense-config-file Densify.ini -v 3
20:48:49 [App     ] MapSMtoCores for SM 8.6 is undefined; default to use 64 cores/SM
20:48:49 [App     ] CUDA device 0 initialized: NVIDIA GeForce RTX 3060 Ti (compute capability 8.6; memory 7.78GB)
20:48:49 [App     ] Scene loaded (5ms):
	19 images (19 calibrated) with a total of 416.89 MPixels (21.94 MPixels/image)
	6482 points, 0 vertices, 0 faces
20:48:49 [App     ] error: too few images to be a tower: '19'
20:48:49 [App     ] Point-cloud composed of 6482 points with:
 - points info:
	1887 points inside ROI (29.11%)
	 inside ROI track length: 2 min / 3.28087 mean (1.39024 std) / 9 max
	outside ROI track length: 2 min / 2.48335 mean (0.734873 std) / 7 max
 - visibility info (17602 views - 2.72 views/point):
	        0 points with 1- views (0.00%)
	     3567 points with 2  views (55.03%)
	     1888 points with 3  views (29.13%)
	     1027 points with 4+ views (15.84%)
	2 min / 2.71552 mean (1.03766 std) / 9 max
20:48:50 [App     ] K18 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K10 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K9 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K15 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K8 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K6 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K13 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K12 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K11 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K0 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K14 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K4 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K16 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K2 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K17 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:50 [App     ] K7 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:51 [App     ] K1 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:51 [App     ] K3 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:51 [App     ] K5 = 
 1655.7214     0.0000  1279.7132 
    0.0000  1655.7214   816.6619 
    0.0000     0.0000     1.0000 

20:48:51 [App     ] Preparing images for dense reconstruction completed: 19 images (1s408ms)
20:48:51 [App     ] Reference image  14 sees 11 views:  12(219pts,2.05scl)  11(172pts,2.09scl)   9(167pts,2.07scl)  15(157pts,1.05scl)  10(61pts,2.08scl)  17(42pts,1.94scl)   2(56pts,2.47scl)  16(40pts,1.01scl)   7(59pts,2.79scl)   0(24pts,3.59scl)   1(11pts,3.38scl) (383 shared points)
20:48:51 [App     ] Reference image   6 sees 5 views:   5(419pts,0.98scl)   7(240pts,0.97scl)   3(136pts,1.28scl)   4(115pts,1.25scl)  13(6pts,0.79scl) (571 shared points)
20:48:51 [App     ] Reference image   6 sees 5 views:   5(419pts,0.98scl)   7(240pts,0.97scl)   3(136pts,1.28scl)   4(115pts,1.25scl)  13(6pts,0.79scl) (571 shared points)
20:48:51 [App     ] Reference image   6 sees 5 views:   5(419pts,0.98scl)   7(240pts,0.97scl)   3(136pts,1.28scl)   4(115pts,1.25scl)  13(6pts,0.79scl) (571 shared points)
20:48:51 [App     ] Reference image   7 sees 16 views:   6(240pts,1.04scl)   5(132pts,0.94scl)   3(88pts,1.11scl)   4(84pts,1.19scl)  14(59pts,0.36scl)  13(20pts,0.83scl)  12(24pts,0.78scl)  11(16pts,0.94scl)   2(11pts,2.43scl)  10(6pts,1.33scl)  17(6pts,1.10scl)   9(7pts,0.83scl)  15(26pts,0.39scl)   0(5pts,2.10scl)   1(3pts,2.31scl)  16(13pts,0.39scl) (467 shared points)
20:48:51 [App     ] Reference image  18 sees 10 views:  17(540pts,0.98scl)   8(149pts,1.21scl)  11(460pts,1.09scl)   1(49pts,1.71scl)  12(59pts,1.10scl)   2(30pts,1.23scl)   0(21pts,1.98scl)  10(30pts,1.14scl)  13(22pts,0.87scl)   9(5pts,1.14scl) (697 shared points)
20:48:51 [App     ] Reference image  10 sees 16 views:   2(594pts,1.26scl)   9(491pts,0.99scl)   0(177pts,1.67scl)  11(393pts,0.97scl)   1(135pts,1.71scl)  17(196pts,0.87scl)  15(153pts,0.49scl)  16(132pts,0.53scl)  12(136pts,0.94scl)  13(82pts,0.60scl)  14(61pts,0.48scl)  18(30pts,0.89scl)   3(23pts,1.35scl)   4(20pts,1.34scl)   8(4pts,1.08scl)   7(6pts,0.87scl) (1000 shared points)
20:48:51 [App     ] Reference image  13 sees 13 views:  11(158pts,1.60scl)   9(98pts,1.63scl)  17(132pts,1.34scl)  10(82pts,1.68scl)   0(51pts,2.76scl)   2(64pts,2.13scl)   1(40pts,2.85scl)  12(22pts,1.38scl)  18(22pts,1.15scl)   7(20pts,1.22scl)   8(10pts,1.32scl)   6(6pts,1.29scl)   5(4pts,1.24scl) (265 shared points)
20:48:51 [App     ] Reference image   4 sees 8 views:   3(1017pts,0.97scl)   6(115pts,0.86scl)   5(104pts,0.89scl)   7(84pts,0.98scl)   2(31pts,1.41scl)  10(19pts,0.84scl)   9(13pts,0.63scl)  16(10pts,0.39scl) (1061 shared points)
20:48:51 [App     ] Reference image   3 sees 8 views:   4(1016pts,1.04scl)   6(136pts,0.85scl)   5(119pts,0.86scl)   7(88pts,1.08scl)   2(35pts,1.45scl)  10(23pts,0.77scl)   9(19pts,0.67scl)  16(14pts,0.39scl) (1090 shared points)
20:48:51 [App     ] Reference image  15 sees 11 views:   9(288pts,2.07scl)  10(153pts,2.04scl)  16(431pts,1.03scl)  11(139pts,2.03scl)  12(126pts,1.95scl)  14(157pts,0.96scl)   2(111pts,2.50scl)   0(28pts,3.43scl)   1(15pts,3.54scl)  17(11pts,1.91scl)   7(26pts,2.59scl) (686 shared points)
20:48:51 [App     ] Reference image   1 sees 14 views:   8(478pts,0.62scl)   0(529pts,0.95scl)   2(123pts,0.68scl)  11(187pts,0.55scl)  10(135pts,0.59scl)  12(94pts,0.54scl)  17(115pts,0.48scl)  18(50pts,0.60scl)   9(77pts,0.57scl)  13(40pts,0.35scl)  15(14pts,0.28scl)  14(11pts,0.30scl)  16(3pts,0.30scl)   7(3pts,0.48scl) (902 shared points)
20:48:51 [App     ] Reference image   8 sees 10 views:   1(477pts,1.62scl)   0(342pts,1.56scl)  11(253pts,0.86scl)   2(183pts,0.80scl)  17(219pts,0.77scl)  18(149pts,0.84scl)  12(135pts,0.90scl)   9(5pts,0.87scl)  10(4pts,0.93scl)  13(10pts,0.80scl) (1040 shared points)
20:48:51 [App     ] Reference image   0 sees 14 views:   1(528pts,1.06scl)   8(342pts,0.65scl)   2(175pts,0.68scl)  11(226pts,0.56scl)  12(138pts,0.55scl)  10(177pts,0.60scl)  17(128pts,0.50scl)   9(119pts,0.58scl)  13(50pts,0.37scl)  18(21pts,0.52scl)  15(27pts,0.29scl)  16(14pts,0.29scl)  14(24pts,0.28scl)   7(5pts,0.53scl) (852 shared points)
20:48:51 [App     ] Reference image   2 sees 16 views:  10(594pts,0.80scl)  11(363pts,0.85scl)   9(364pts,0.80scl)   8(180pts,1.27scl)   0(179pts,1.57scl)   1(124pts,1.54scl)  17(183pts,0.77scl)  12(151pts,0.90scl)  15(110pts,0.40scl)  16(86pts,0.42scl)  18(30pts,0.83scl)  14(56pts,0.41scl)   3(36pts,0.74scl)  13(64pts,0.48scl)   4(31pts,0.75scl)   7(11pts,0.47scl) (1017 shared points)
20:48:51 [App     ] Reference image  11 sees 14 views:  17(1345pts,0.88scl)   2(365pts,1.20scl)  12(982pts,0.99scl)   8(253pts,1.17scl)   0(228pts,1.81scl)   1(185pts,1.84scl)   9(481pts,1.00scl)  10(393pts,1.04scl)  18(459pts,0.92scl)  14(172pts,0.48scl)  13(155pts,0.63scl)  15(138pts,0.50scl)  16(39pts,0.52scl)   7(16pts,1.12scl) (2253 shared points)
20:48:51 [App     ] Reference image  12 sees 14 views:  11(977pts,1.02scl)  17(663pts,0.88scl)   0(138pts,1.83scl)   2(150pts,1.15scl)   8(135pts,1.12scl)   9(328pts,1.03scl)   1(95pts,1.85scl)  14(219pts,0.49scl)  15(124pts,0.52scl)  10(136pts,1.07scl)  18(57pts,0.91scl)  16(23pts,0.52scl)  13(22pts,0.72scl)   7(24pts,1.30scl) (1325 shared points)
20:48:51 [App     ] Reference image  17 sees 13 views:  11(1351pts,1.13scl)  12(668pts,1.14scl)   2(181pts,1.33scl)   8(219pts,1.30scl)  18(541pts,1.02scl)   9(216pts,1.10scl)  10(195pts,1.16scl)   0(130pts,2.03scl)   1(114pts,2.08scl)  13(132pts,0.75scl)  14(42pts,0.52scl)  15(11pts,0.52scl)   7(6pts,0.92scl) (1740 shared points)
20:48:51 [App     ] Reference image   9 sees 16 views:   2(362pts,1.26scl)  10(488pts,1.01scl)  11(480pts,1.01scl)  15(287pts,0.48scl)  12(326pts,0.98scl)   0(119pts,1.73scl)  17(213pts,0.91scl)  16(236pts,0.51scl)  14(165pts,0.49scl)   1(77pts,1.75scl)  13(96pts,0.62scl)   8(5pts,1.16scl)   3(19pts,1.52scl)   4(14pts,1.62scl)   7(7pts,1.24scl)  18(5pts,0.88scl) (1122 shared points)
20:48:51 [App     ] Selecting images for dense reconstruction completed: 19 images (328ms)
Geometric-consistent estimated depth-maps 15 (78.95%, 32s, ETA 8s)... Segmentation fault (core dumped)

Trying to run the command for other scenes:

  • scene_0001.mvs works
  • scene_0002.mvs works
  • scene_0003.mvs crashes with 0 images (0 calibrated) with a total of 0.00 MPixels (-nan MPixels/image) (probably a different issue, looks like the scene is empty)
  • scene_0004.mvs works

Looking at what command (2) produces:

20:24:04 [App     ] OpenMVS x64 v2.2.0
20:24:04 [App     ] Build date: Jan 25 2024, 10:11:25
20:24:04 [App     ] CPU: AMD Ryzen 7 1700 Eight-Core Processor           (16 cores)
20:24:04 [App     ] RAM: 62.74GB Physical Memory 32.00GB Virtual Memory
20:24:04 [App     ] OS: Linux 5.15.146-1-MANJARO (x86_64)
20:24:05 [App     ] Disk: 216.71GB (960.17GB) space
20:24:05 [App     ] SSE & AVX compatible CPU & OS detected
20:24:05 [App     ] Command line: DensifyPointCloud scene.mvs --resolution-level 1 --iters 4 --cuda-device -1 --sub-scene-area 20000000
20:24:05 [App     ] MapSMtoCores for SM 8.6 is undefined; default to use 64 cores/SM
20:24:05 [App     ] CUDA device 0 initialized: NVIDIA GeForce RTX 3060 Ti (compute capability 8.6; memory 7.78GB)
20:24:05 [App     ] Camera model loaded: platform 0; camera  0; f 1.201x1.201; poses 2
20:24:05 [App     ] Camera model loaded: platform 1; camera  0; f 1.144x1.144; poses 1
20:24:05 [App     ] Camera model loaded: platform 2; camera  0; f 1.402x1.402; poses 1
20:24:05 [App     ] Camera model loaded: platform 3; camera  0; f 0.680x0.680; poses 1
20:24:05 [App     ] Camera model loaded: platform 4; camera  0; f 1.305x1.305; poses 2
20:24:05 [App     ] Camera model loaded: platform 5; camera  0; f 1.407x1.407; poses 3
20:24:05 [App     ] Camera model loaded: platform 6; camera  0; f 1.014x1.014; poses 2
20:24:05 [App     ] Camera model loaded: platform 7; camera  0; f 1.976x1.976; poses 1
20:24:05 [App     ] Camera model loaded: platform 8; camera  0; f 0.873x0.873; poses 1
20:24:05 [App     ] Camera model loaded: platform 9; camera  0; f 1.949x1.949; poses 7
20:24:05 [App     ] Camera model loaded: platform 10; camera  0; f 0.647x0.647; poses 837
20:24:05 [App     ] Camera model loaded: platform 11; camera  0; f 1.486x1.486; poses 18
20:24:05 [App     ] Camera model loaded: platform 12; camera  0; f 0.728x0.728; poses 1
20:24:05 [App     ] Camera model loaded: platform 13; camera  0; f 2.121x2.121; poses 39
20:24:05 [App     ] Camera model loaded: platform 14; camera  0; f 1.703x1.703; poses 2
20:24:05 [App     ] Camera model loaded: platform 15; camera  0; f 1.071x1.071; poses 2
20:24:05 [App     ] Camera model loaded: platform 16; camera  0; f 1.642x1.642; poses 4
20:24:05 [App     ] Camera model loaded: platform 17; camera  0; f 0.821x0.821; poses 6
20:24:05 [App     ] Camera model loaded: platform 18; camera  0; f 0.990x0.990; poses 1
20:24:05 [App     ] Camera model loaded: platform 19; camera  0; f 0.896x0.896; poses 4
20:24:05 [App     ] Camera model loaded: platform 20; camera  0; f 1.769x1.769; poses 2
20:24:05 [App     ] Camera model loaded: platform 21; camera  0; f 0.938x0.938; poses 1
20:24:06 [App     ] Camera model loaded: platform 22; camera  0; f 0.780x0.780; poses 8
20:24:06 [App     ] Scene loaded from interface format (849ms):
	946 images (946 calibrated) with a total of 20426.95 MPixels (21.59 MPixels/image)
	311337 points, 0 vertices, 0 faces
20:24:06 [App     ] Found a camera not pointing towards the scene center; the scene will be considered unbounded (no ROI)
20:24:06 [App     ] error: does not seem to be a tower: X(44.74), Y(2183.39), Z(10851.75)
20:24:07 [App     ] error: opening file 'depth0966.dmap' for reading depth-data
20:24:16 [App     ] error: opening file 'depth0896.dmap' for reading depth-data
20:24:20 [App     ] error: opening file 'depth0875.dmap' for reading depth-data
20:24:20 [App     ] error: opening file 'depth0873.dmap' for reading depth-data
20:24:22 [App     ] error: opening file 'depth0843.dmap' for reading depth-data
20:24:22 [App     ] error: opening file 'depth0842.dmap' for reading depth-data
20:24:24 [App     ] error: opening file 'depth0834.dmap' for reading depth-data
20:24:24 [App     ] error: opening file 'depth0833.dmap' for reading depth-data
20:24:40 [App     ] error: opening file 'depth0712.dmap' for reading depth-data
20:24:40 [App     ] error: opening file 'depth0711.dmap' for reading depth-data
20:24:40 [App     ] error: opening file 'depth0706.dmap' for reading depth-data
20:25:03 [App     ] error: opening file 'depth0221.dmap' for reading depth-data
20:25:03 [App     ] error: opening file 'depth0216.dmap' for reading depth-data
20:25:05 [App     ] error: opening file 'depth0193.dmap' for reading depth-data
20:25:05 [App     ] error: opening file 'depth0192.dmap' for reading depth-data
20:25:06 [App     ] error: opening file 'depth0180.dmap' for reading depth-data
20:25:11 [App     ] error: opening file 'depth0046.dmap' for reading depth-data
20:25:12 [App     ] error: opening file 'depth0035.dmap' for reading depth-data
20:25:12 [App     ] error: opening file 'depth0034.dmap' for reading depth-data
20:25:21 [App     ] error: opening file 'depth0113.dmap' for reading depth-data
20:25:32 [App     ] error: opening file 'depth0365.dmap' for reading depth-data
20:25:32 [App     ] error: opening file 'depth0369.dmap' for reading depth-data
20:25:32 [App     ] error: opening file 'depth0370.dmap' for reading depth-data
20:25:38 [App     ] error: opening file 'depth0398.dmap' for reading depth-data
20:25:38 [App     ] error: opening file 'depth0399.dmap' for reading depth-data
20:25:38 [App     ] error: opening file 'depth0400.dmap' for reading depth-data
20:25:38 [App     ] error: opening file 'depth0402.dmap' for reading depth-data
20:25:56 [App     ] Point-cloud saved: 8 points (0ms)
20:26:03 [App     ] Scene split (2e+07 max-area): 50 chunks (1m57s227ms)
20:26:04 [App     ] Scene saved (3ms):
	19 images (19 calibrated)
	6482 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (2ms):
	13 images (13 calibrated)
	3716 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (1ms):
	7 images (7 calibrated)
	2289 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (0ms):
	0 images (0 calibrated)
	0 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (8ms):
	20 images (20 calibrated)
	6352 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (14ms):
	27 images (27 calibrated)
	10067 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (15ms):
	35 images (35 calibrated)
	10579 points, 0 vertices, 0 faces
20:26:04 [App     ] Scene saved (10ms):
	29 images (29 calibrated)
	8943 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (26ms):
	53 images (53 calibrated)
	21698 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (3ms):
	16 images (16 calibrated)
	5855 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (13ms):
	22 images (22 calibrated)
	10683 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (15ms):
	26 images (26 calibrated)
	11753 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (12ms):
	32 images (32 calibrated)
	10342 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (7ms):
	16 images (16 calibrated)
	5933 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (10ms):
	21 images (21 calibrated)
	7328 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (21ms):
	45 images (45 calibrated)
	16404 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (16ms):
	38 images (38 calibrated)
	14221 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (6ms):
	11 images (11 calibrated)
	5917 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (7ms):
	18 images (18 calibrated)
	5763 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (8ms):
	24 images (24 calibrated)
	6549 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (6ms):
	18 images (18 calibrated)
	4920 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (12ms):
	32 images (32 calibrated)
	10797 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (4ms):
	28 images (28 calibrated)
	6334 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (2ms):
	12 images (12 calibrated)
	2794 points, 0 vertices, 0 faces
20:26:05 [App     ] Scene saved (2ms):
	12 images (12 calibrated)
	3380 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (6ms):
	28 images (28 calibrated)
	9537 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (8ms):
	36 images (36 calibrated)
	12608 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (4ms):
	21 images (21 calibrated)
	8342 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (14ms):
	37 images (37 calibrated)
	12316 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (28ms):
	51 images (51 calibrated)
	20730 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (5ms):
	34 images (34 calibrated)
	9340 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (14ms):
	41 images (41 calibrated)
	19058 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (12ms):
	28 images (28 calibrated)
	10443 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (12ms):
	28 images (28 calibrated)
	10904 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (5ms):
	13 images (13 calibrated)
	4623 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (1ms):
	9 images (9 calibrated)
	1137 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (24ms):
	38 images (38 calibrated)
	18420 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (6ms):
	10 images (10 calibrated)
	4881 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (11ms):
	29 images (29 calibrated)
	8772 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (8ms):
	17 images (17 calibrated)
	7483 points, 0 vertices, 0 faces
20:26:06 [App     ] Scene saved (28ms):
	47 images (47 calibrated)
	21315 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (3ms):
	12 images (12 calibrated)
	4510 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (23ms):
	52 images (52 calibrated)
	20515 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (15ms):
	28 images (28 calibrated)
	14725 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (6ms):
	21 images (21 calibrated)
	5524 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (1ms):
	15 images (15 calibrated)
	2664 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (2ms):
	10 images (10 calibrated)
	3331 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (1ms):
	7 images (7 calibrated)
	894 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (9ms):
	39 images (39 calibrated)
	7006 points, 0 vertices, 0 faces
20:26:07 [App     ] Scene saved (10ms):
	53 images (53 calibrated)
	18547 points, 0 vertices, 0 faces
20:26:07 [App     ] MEMORYINFO: {
20:26:07 [App     ] 	VmPeak:	 6554220 kB
20:26:08 [App     ] 	VmSize:	 5320520 kB
20:26:08 [App     ] } ENDINFO

suggests that certain depth maps are malformed, which could perhaps be a reason for the issue?
@xiaoxiae
Copy link
Author

xiaoxiae commented Feb 1, 2024

The depthmaps in question are actually all missing, not malformed.

@cdcseacave
Copy link
Owner

pls check the log for the first step to see why some depth-maps were not estimated; I'd appreciate if you can help debugging this

@xiaoxiae
Copy link
Author

xiaoxiae commented Feb 2, 2024

This is the log of the first command:

DensifyPointCloud-2402011542308B498E.log

The issues seem to persist, I don't see why they weren't generated.

I tried re-running the pipeline overnight (with --resolution-level 2) and encountered the same issue. I will do so now again with -v 3 and see if it reveals anything.

@xiaoxiae
Copy link
Author

xiaoxiae commented Feb 2, 2024 •
edited

Rerunning the pipeline with

  1. DensifyPointCloud scene.mvs --resolution-level 2 --iters 4 -v 3 --cuda-device -1 --fusion-mode 1
  2. DensifyPointCloud scene.mvs --resolution-level 2 --iters 4 -v 3 --cuda-device -1 --sub-scene-area 20000000
  3. DensifyPointCloud scene_0000.mvs --resolution-level 2 --iters 4 -v 3 --cuda-device -1 --dense-config-file Densify.ini

segfaults on the 3rd command (this time with CUDA error: invalid argument (code 1)) and yields the following log files:

  1. DensifyPointCloud-2402021007118B4721.log
  2. DensifyPointCloud-2402021340068B475B.log
  3. DensifyPointCloud-2402021345248B475F.log

The last line before the crash is

13:48:21 [App     ] Reference image  22 paired with 4 views: 873(1.74scl) 875(1.90scl) 879(1.00scl) 874(1.93scl) (609 shared points)

which likely crashes because depth maps 873 and 875 are missing.

Looking over the logs, I don't see anything overtly suspicious for why the depth maps are not created.

I'm now going to run badblocks on the disk storing the depthmaps to ensure this is not a hardware issue. Besides that, I'm out of ideas and am happy to try out whatever you suggest!

@xiaoxiae
Copy link
Author

xiaoxiae commented Feb 2, 2024 •
edited

Digging further, it seems that the InterfaceCOLMAP (run before 1, 2 and 3) only exports a subset of images (946 out of the 972 total ones):

10:07:07 [App     ] OpenMVS x64 v2.3.0
10:07:08 [App     ] Build date: Feb  2 2024, 00:09:03
10:07:08 [App     ] CPU: AMD Ryzen 7 1700 Eight-Core Processor           (16 cores)
10:07:08 [App     ] RAM: 62.74GB Physical Memory 32.00GB Virtual Memory
10:07:08 [App     ] OS: Linux 5.15.146-1-MANJARO (x86_64), which is likely because some images weren't matched. Weirdly enough, 
10:07:08 [App     ] Disk: 199.47GB (960.17GB) space
10:07:09 [App     ] SSE & AVX compatible CPU & OS detected
10:07:09 [App     ] Command line: InterfaceCOLMAP -i /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs -o /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/scene.mvs --image-folder /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/images
10:07:09 [App     ] Reading cameras: /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/sparse/cameras.bin
10:07:09 [App     ] Reading images: /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/sparse/images.bin
10:07:09 [App     ] Reading points: /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/sparse/points3D.bin
10:07:11 [App     ] Reading patch-match configuration: /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/stereo/patch-match.cfg
10:07:11 [App     ] Reading depth-maps/normal-maps: /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/stereo/depth_maps/ and /app/dst/new-wall-pipeline-smichoff/03-LargeDenseCheckpoint/project/mvs/stereo/normal_maps/
10:07:11 [App     ] Exported data: 946 images, 311337 points (2s467ms)
10:07:11 [App     ] MEMORYINFO: {
10:07:11 [App     ] 	VmPeak:	  377528 kB
10:07:11 [App     ] 	VmSize:	  345276 kB
10:07:11 [App     ] } ENDINFO

Weirdly enough, the difference is 972 - 946 = 26 but the reading error happens for 27 depthmaps:

13:45:01 [App     ] error: opening file 'depth0966.dmap' for reading depth-data
13:45:02 [App     ] error: opening file 'depth0896.dmap' for reading depth-data
13:45:02 [App     ] error: opening file 'depth0875.dmap' for reading depth-data
13:45:02 [App     ] error: opening file 'depth0873.dmap' for reading depth-data
13:45:03 [App     ] error: opening file 'depth0843.dmap' for reading depth-data
13:45:03 [App     ] error: opening file 'depth0842.dmap' for reading depth-data
13:45:03 [App     ] error: opening file 'depth0834.dmap' for reading depth-data
13:45:03 [App     ] error: opening file 'depth0833.dmap' for reading depth-data
13:45:05 [App     ] error: opening file 'depth0712.dmap' for reading depth-data
13:45:05 [App     ] error: opening file 'depth0711.dmap' for reading depth-data
13:45:05 [App     ] error: opening file 'depth0706.dmap' for reading depth-data
13:45:08 [App     ] error: opening file 'depth0221.dmap' for reading depth-data
13:45:08 [App     ] error: opening file 'depth0216.dmap' for reading depth-data
13:45:08 [App     ] error: opening file 'depth0193.dmap' for reading depth-data
13:45:08 [App     ] error: opening file 'depth0192.dmap' for reading depth-data
13:45:08 [App     ] error: opening file 'depth0180.dmap' for reading depth-data
13:45:09 [App     ] error: opening file 'depth0046.dmap' for reading depth-data
13:45:10 [App     ] error: opening file 'depth0035.dmap' for reading depth-data
13:45:10 [App     ] error: opening file 'depth0034.dmap' for reading depth-data
13:45:12 [App     ] error: opening file 'depth0113.dmap' for reading depth-data
13:45:14 [App     ] error: opening file 'depth0365.dmap' for reading depth-data
13:45:14 [App     ] error: opening file 'depth0369.dmap' for reading depth-data
13:45:14 [App     ] error: opening file 'depth0370.dmap' for reading depth-data
13:45:15 [App     ] error: opening file 'depth0398.dmap' for reading depth-data
13:45:15 [App     ] error: opening file 'depth0399.dmap' for reading depth-data
13:45:15 [App     ] error: opening file 'depth0400.dmap' for reading depth-data
13:45:15 [App     ] error: opening file 'depth0402.dmap' for reading depth-data

@xiaoxiae
Copy link
Author

xiaoxiae commented Feb 4, 2024 •
edited

badblocks on the disk in question produced no errors so the problem likely isn't in the disk.

@xiaoxiae
Copy link
Author

Ran MemTest86 to check for RAM issues (possibly corrupting the DMAPs on export), no fails there.

@xiaoxiae
Copy link
Author

Rerunning the code on the machine the issue was present on and also on a different one to see if the problem is with the data.

@xiaoxiae
Copy link
Author

The errors occur for the same depth maps when run multiple times.

@xiaoxiae
Copy link
Author

I'm at a loss for what to try at this point, let me know what I should do if you have any ideas.

@xiaoxiae
Copy link
Author

Found a minimal reproducible example.

Taking this COLMAP project: 1-colmap-input.zip
I ran COLMAP's image_undistorter, producing the following: 2-after-undistortion.zip

Using OpenMVS, I ran

  1. InterfaceCOLMAP -i . -o scene.mvs --image-folder images
  2. DensifyPointCloud scene.mvs --resolution-level 2 --iters 4 --cuda-device -1 --fusion-mode 1
  3. DensifyPointCloud scene.mvs --resolution-level 2 --iters 4 --cuda-device -1 --sub-scene-area 20000000
  4. create Densify.ini with Optimize = 0
  5. DensifyPointCloud scene_0000.mvs --resolution-level 2 --iters 4 --cuda-device -1 --dense-config-file Densify.ini

The last step crashes with

malloc(): unsorted double linked list corrupted
Aborted (core dumped)

Simply running DensifyPointCloud scene.mvs --resolution-level 2 --iters 4 --cuda-device -1 works perfectly fine and produces the correct model.

Here are all produced logs and files: 3-after-crash.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cdcseacave @xiaoxiae