Error with ibtracert/Route command and network examples

[rhettdsouza13]

When simulating the network using the config file net.2sw2path4hca provided in the list of examples we get an error that results in the network being created but lids not being initialized properly. Therefore, we are unable to route packages through the network. With the following error when running the Route command:

bad lid value. Usage: route from-lid to-lid

Output of $ ./ibsim/ibsim -s net-examples/net.2sw2path4hca:

parsing: net-examples/net.2sw2path4hca
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [1]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [1]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [3]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [5]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [1]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [1]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [3]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 2 : [5]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 1 : [1]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 1 : [1]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 1 : [2]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
ibwarn: [29566] parse_port_connection_data: Parsing 1 : [2]
ibwarn: [29566] parse_port_connection_data: cannot parse remote lid and connection type
net-examples/net.2sw2path4hca: parsed 43 lines
########################
Network simulator ready.
MaxNetNodes    = 2048
MaxNetSwitches = 256
MaxNetPorts    = 13312
MaxLinearCap   = 30720
MaxMcastCap    = 1024

I'm unsure as to whether the configuration example files provided are correct themselves, as the output of ibnetdiscover shows that no lids have been created or initialized.

What are we missing here? Do we need to provide the correct information in the net config files? What is the appropriate format for those files? What's the appropriate technique for routing packages through the simulated network?

Output of ibnetdiscover:

ibwarn: [29597] sim_connect: attached as client 0 at node "Switch1"
#
# Topology file: generated on Mon Mar 25 01:45:09 2019
#
# Initiated from node 0000000000200000 port 0000000000200000

vendid=0x0
devid=0x0
sysimgguid=0x200001
switchguid=0x200001(200001)
Switch	8 "S-0000000000200001"		# "Switch2" base port 0 lid 0 lmc 0
[1]	"H-0000000000100003"[1](100004) 		# "Hca2" lid 0 4xSDR
[2]	"H-0000000000100009"[1](10000a) 		# "Hca4" lid 0 4xSDR
[3]	"S-0000000000200000"[3]		# "Switch1" lid 0 4xSDR
[5]	"S-0000000000200000"[5]		# "Switch1" lid 0 4xSDR

vendid=0x0
devid=0x0
sysimgguid=0x200000
switchguid=0x200000(200000)
Switch	8 "S-0000000000200000"		# "Switch1" base port 0 lid 0 lmc 0
[1]	"H-0000000000100000"[1](100001) 		# "Hca1" lid 0 4xSDR
[2]	"H-0000000000100006"[1](100007) 		# "Hca3" lid 0 4xSDR
[3]	"S-0000000000200001"[3]		# "Switch2" lid 0 4xSDR
[5]	"S-0000000000200001"[5]		# "Switch2" lid 0 4xSDR

vendid=0x0
devid=0x0
sysimgguid=0x100009
caguid=0x100009
Ca	2 "H-0000000000100009"		# "Hca4"
[1](10000a) 	"S-0000000000200001"[2]		# lid 0 lmc 0 "Switch2" lid 0 4xSDR

vendid=0x0
devid=0x0
sysimgguid=0x100003
caguid=0x100003
Ca	2 "H-0000000000100003"		# "Hca2"
[1](100004) 	"S-0000000000200001"[1]		# lid 0 lmc 0 "Switch2" lid 0 4xSDR

vendid=0x0
devid=0x0
sysimgguid=0x100006
caguid=0x100006
Ca	2 "H-0000000000100006"		# "Hca3"
[1](100007) 	"S-0000000000200000"[2]		# lid 0 lmc 0 "Switch1" lid 0 4xSDR

vendid=0x0
devid=0x0
sysimgguid=0x100000
caguid=0x100000
Ca	2 "H-0000000000100000"		# "Hca1"
[1](100001) 	"S-0000000000200000"[1]		# lid 0 lmc 0 "Switch1" lid 0 4xSDR

Thank you so much for your help and support.

[hnrose]

For LID routing to work, LIDs must be assigned to the end ports and the routing tables (LFTs) setup. This is the job of the SM (subnet manager) - OpenSM. After running OpenSM, these commands should work properly.

[rhettdsouza13]

Thanks so much for the response!
As we are new to ibsim and opensm, could you give us an example of how we may run opensm after
we run the command ./ibsim/ibsim -s net-examples/net.1.

As we are trying to do the same, we get the error as No local ports detected!
Are we doing it the right way? Thanks so much for your help.
(Note: We even attempted to do it with the run_opensm.sh script with the same error)

Ouput of the command with high verbosity:

$ sudo opensm -V -f -
-------------------------------------------------
OpenSM 3.3.19
Command Line Arguments:
 Big V selected
 Log File: -
-------------------------------------------------
Mar 26 01:05:17 996894 [A46C2700] 0x7f -> OpenSM 3.3.19
Mar 26 01:05:17 997040 [A46C2700] 0x80 -> OpenSM 3.3.19
Mar 26 01:05:17 997076 [A46C2700] 0x10 -> osm_opensm_init: [
Mar 26 01:05:17 997539 [A46C2700] 0x10 -> osm_db_init: [
Mar 26 01:05:17 997570 [A46C2700] 0x10 -> osm_db_init: ]
Mar 26 01:05:17 997586 [A46C2700] 0x10 -> osm_db_domain_init: [
Mar 26 01:05:17 997631 [A46C2700] 0x10 -> osm_db_domain_init: ]
Mar 26 01:05:17 997642 [A46C2700] 0x10 -> osm_db_restore: [
Mar 26 01:05:17 997678 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000100004 value:0x0000000000000000
Mar 26 01:05:17 997695 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200000 value:0x0000000000000000
Mar 26 01:05:17 997708 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200001 value:0x0000000000000000
Mar 26 01:05:17 997719 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000100001 value:0x0000000000000000
Mar 26 01:05:17 997738 [A46C2700] 0x10 -> osm_db_restore: ]
Mar 26 01:05:17 997748 [A46C2700] 0x10 -> subn_validate_g2m: [
Mar 26 01:05:17 997774 [A46C2700] 0x10 -> subn_validate_g2m: ]
Mar 26 01:05:17 997785 [A46C2700] 0x10 -> osm_db_domain_init: [
Mar 26 01:05:17 997808 [A46C2700] 0x10 -> osm_db_domain_init: ]
Mar 26 01:05:17 997818 [A46C2700] 0x10 -> osm_db_restore: [
Mar 26 01:05:17 997853 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200001:5 value:0x0000000000200000:5
Mar 26 01:05:17 997873 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200000:1 value:0x0000000000100001:1
Mar 26 01:05:17 997890 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000100004:1 value:0x0000000000200001:1
Mar 26 01:05:17 997907 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200000:3 value:0x0000000000200001:3
Mar 26 01:05:17 997923 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000100001:1 value:0x0000000000200000:1
Mar 26 01:05:17 997941 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200001:1 value:0x0000000000100004:1
Mar 26 01:05:17 997957 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200000:5 value:0x0000000000200001:5
Mar 26 01:05:17 997974 [A46C2700] 0x08 -> osm_db_restore: Got key:0x0000000000200001:3 value:0x0000000000200000:3
Mar 26 01:05:17 997996 [A46C2700] 0x10 -> osm_db_restore: ]
Mar 26 01:05:17 998011 [A46C2700] 0x10 -> subn_validate_neighbor: [
Mar 26 01:05:17 998042 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000200001, port 3
Mar 26 01:05:17 998061 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000200001, port 1
Mar 26 01:05:17 998082 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000200000, port 5
Mar 26 01:05:17 998100 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000100004, port 1
Mar 26 01:05:17 998118 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000200000, port 3
Mar 26 01:05:17 998136 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000100001, port 1
Mar 26 01:05:17 998155 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000200000, port 1
Mar 26 01:05:17 998175 [A46C2700] 0x08 -> subn_validate_neighbor: Validating neighbor for guid:0x0000000000200001, port 5
Mar 26 01:05:17 998193 [A46C2700] 0x10 -> subn_validate_neighbor: ]
Mar 26 01:05:17 998209 [A46C2700] 0x10 -> osm_vendor_new: [
Mar 26 01:05:17 998228 [A46C2700] 0x10 -> osm_vendor_init: [
Mar 26 01:05:17 998316 [A46C2700] 0x02 -> osm_vendor_init: 1000 pending umads specified
Mar 26 01:05:17 998354 [A46C2700] 0x10 -> osm_vendor_init: ]
Mar 26 01:05:17 998367 [A46C2700] 0x10 -> osm_vendor_new: ]
Mar 26 01:05:17 998379 [A46C2700] 0x10 -> osm_opensm_init: ]
Mar 26 01:05:17 998402 [A46C2700] 0x10 -> osm_vendor_get_all_port_attr: [
Mar 26 01:05:17 998414 [A46C2700] 0x10 -> osm_vendor_get_all_port_attr: ]

No local ports detected!
Mar 26 01:05:17 998456 [A46C2700] 0x10 -> clear_madw: [
Mar 26 01:05:17 998477 [A46C2700] 0x10 -> clear_madw: ]

[hnrose]

You should read the ibsim README. You are running OpenSM on real hardware but there are no HCAs so it reports No local ports detected. After starting the simulator, you need to do something like:
LD_PRELOAD=./umad2sim/libumad2sim.so opensm -f -
to run this on the simulator. You can change the point of attachment if you don't like what OpenSM chooses by default - see ibsim README.

[rhettdsouza13]

Thank you so much!
I had exported LD_PRELOAD, expecting it to work as it would be part of the variable but I tried running it in a command-like fashion as you mentioned and it worked perfectly.
Thanks again for your help!