Data Pre-processing

[shenben]

After downloading the MOLIERE_2016 dataset, I extracted the files to obtain the following graph format files: MOLIERE_2016_duplicate_edges.mtx, MOLIERE_2016_labels.txt, and MOLIERE_2016.mtx. I am wondering how I preprocess the files or convert them to a specific format such as MOLIERE.adj and MOLIERE.idx.
Clearly, just how to generate a directed or symmetric directed graph for the following execution commands you mentioned in the README.

Server Command

./main -m server -e /mnt/graph1/fam-graph/MOLIERE.adj -t 10

Client Command

./main -m client -k bfs -i /mnt/graphs/fam-graph/MOLIERE.idx -t 10 --start-vertex 1

Nothing would be better if you could provide a proper script or share me with a reference link for this. Looking forward to your reply. Thank you @danieldzahka in advance!

[danieldzahka]

You can use the tool: util/main.cpp to convert from plain text edge list to the csr binary format (two files, one with .idx the other .adj). The tool expects input of the form:

1 2
1 3
2 4

For a graph with edges: 1 -> 2, 1 -> 3, 2 -> 4. The tool cannot tolerate any lines that aren't in the form of two numbers separated by a space, so for the Moliere data set, you will have to remove all of the leading lines starting with # and the first line after that with three numbers describing the shape of the matrix. The tool also has an option for adding reverse edges to make the graph a symmetric digraph, which you will need to use for the Moliere dataset if I recall correctly.

[shenben]

Thank you for your information!

After preprocessing the dataset well, the framework can still not establish an RDMA connection between the server and the client.

The running error information is as follow:

(DGL) emc_admin@emcsvr01:~/fam-bak/build$ ./src/main -m server -e /home/emc_admin/fam-bak/data/MOLIERE_2016_p.adj -t 10
[2023-02-22 03:07:15.439230] [0x00007f882ab18780] [info]    Starting server
[2023-02-22 03:07:15.439265] [0x00007f882ab18780] [info]    Server IPoIB address: 10.119.47.136 port: 12345
[2023-02-22 03:07:15.439278] [0x00007f882ab18780] [info]    Reading in edgelist
[2023-02-22 03:07:15.439292] [0x00007f882ab18780] [info]    hugepages? false
[2023-02-22 03:07:15.439303] [0x00007f882ab18780] [info]    waiting for connections. interrupt (^C) to exit.
error: ec = rdma_create_event_channel() failed (returned zero/null).
(DGL) emc_admin@emcsvr01:~/fam-bak/build$ ./src/main -m client -k bfs -i /home/emc_admin/fam-bak/data/MOLIERE_2016_p.idx -t 10 --start-vertex 1
[2023-02-22 03:08:32.804341] [0x00007f32b0ade780] [info]    Starting client
[2023-02-22 03:08:32.804391] [0x00007f32b0ade780] [info]    Server IPoIB address: 10.119.47.136 port: 12345
[2023-02-22 03:08:32.804414] [0x00007f32b0ade780] [info]    Index File: /home/emc_admin/fam-bak/data/MOLIERE_2016_p.idx
error: ec = rdma_create_event_channel() failed (returned zero/null).

Is it possible that the code of this version can support RoCE?

emc_admin@emcsvr01:~$ ibv_devices
    device                 node GUID
    ------              ----------------
    roceo3              b22628fffe4fdcce

[danieldzahka]

If RoCE implements the ibverbs interface (including the specific configurations used in the system: RC Transport, and the other qp settings are ok e.g. CQ depths), then it should work. It looks like there is failure in rdma_create_event_channel(). Can you try checking your permissions or maybe just running as root to make sure that isn't the issue.