Exporting to Pandas dataframes
As we are ingesting from a set of dataframes, let's kick off by looking at how to convert back into them. For this Raphtory provides the to_df() function on both the Nodes and Edges.
Node Dataframe
To explore the use of to_df() on the nodes we can first we call the function with default parameters. This exports only the latest property updates and utilises epoch timestamps - the output from this can be seen below.
To demonstrate the flags which can be utilised, we call to_df() again, this time enabling the property history and utilising datetime timestamps. The output for this can also be seen below.
df = traffic_graph.nodes.to_df()
print("--- to_df with default parameters --- ")
print(f"{df}\n")
print()
df = traffic_graph.nodes.to_df(include_property_history=True, convert_datetime=True)
print("--- to_df with property history and datetime conversion ---")
print(f"{df}\n")
Output
--- to_df with default parameters ---
name ... update_history
0 ServerE ... [1693556100000, 1693556400000, 1693556700000]
1 ServerA ... [1693555200000, 1693555500000, 1693556400000]
2 ServerD ... [1693555800000, 1693556100000, 1693557000000]
3 ServerB ... [1693555200000, 1693555500000, 1693555800000, ...
4 ServerC ... [1693555500000, 1693555800000, 1693556400000, ...
[5 rows x 9 columns]
--- to_df with property history and datetime conversion ---
name ... update_history
0 ServerE ... [2023-09-01 08:15:00+00:00, 2023-09-01 08:20:0...
1 ServerA ... [2023-09-01 08:00:00+00:00, 2023-09-01 08:05:0...
2 ServerD ... [2023-09-01 08:10:00+00:00, 2023-09-01 08:15:0...
3 ServerB ... [2023-09-01 08:00:00+00:00, 2023-09-01 08:05:0...
4 ServerC ... [2023-09-01 08:05:00+00:00, 2023-09-01 08:10:0...
[5 rows x 9 columns]
Edge Dataframe
Exporting to an edge dataframe via to_df() works the same as for the nodes. The only difference is by default this will export the property history for each edge, split by edge layer. This is because this function has an alternative flag to explode the edges and view each update individually (which will then ignore the include_property_history flag).
In the below example we first create a subgraph of the monkey interactions, selecting some monkeys we are interested in (ANGELE and FELIPE). This isn't a required step, but is to demonstrate the export working on graph views.
We then call to_df() on the subgraph edges, setting no flags. In the output you can see the property history for each interaction type (layer) between ANGELE and FELIPE.
Finally, we call to_df() again, turning off the property history and exploding the edges. In the output you can see each interaction which occurred between ANGELE and FELIPE.
Info
We have further reduced the graph to only one layer (Grunting-Lipsmacking) to reduce the output size.
subgraph = monkey_graph.subgraph(["ANGELE", "FELIPE"])
df = subgraph.edges.to_df()
print("Interactions between Angele and Felipe:")
print(f"{df}\n")
grunting_graph = subgraph.layer("Grunting-Lipsmacking")
df = grunting_graph.edges.to_df(explode=True)
print("Exploding the grunting-Lipsmacking layer")
print(df)
Output
Interactions between Angele and Felipe:
src ... update_history
0 ANGELE ... [1560419400000, 1560419400000, 1560419460000, ...
1 ANGELE ... [1560422580000, 1560441780000, 1560441780000, ...
2 ANGELE ... [1560855660000]
3 ANGELE ... [1560526320000, 1560855660000, 1561042620000]
4 ANGELE ... [1561720320000]
5 ANGELE ... [1562253540000]
6 FELIPE ... [1560419460000, 1560419520000, 1560419580000, ...
7 FELIPE ... [1562321580000]
8 FELIPE ... [1560526320000, 1561972860000, 1562253540000]
9 FELIPE ... [1560526260000, 1562253540000, 1562321580000]
10 FELIPE ... [1562057520000, 1562671200000]
11 FELIPE ... [1562253540000]
12 FELIPE ... [1560526320000]
13 FELIPE ... [1561110180000]
14 FELIPE ... [1562057520000]
[15 rows x 5 columns]
Exploding the grunting-Lipsmacking layer
src dst layer Weight update_history
0 ANGELE FELIPE Grunting-Lipsmacking 1 1560526320000
1 ANGELE FELIPE Grunting-Lipsmacking 1 1560855660000
2 ANGELE FELIPE Grunting-Lipsmacking 1 1561042620000
3 FELIPE ANGELE Grunting-Lipsmacking 1 1560526320000
4 FELIPE ANGELE Grunting-Lipsmacking 1 1561972860000
5 FELIPE ANGELE Grunting-Lipsmacking 1 1562253540000