Hi I’m Mark T, and I’m currently working as an intern in MarketScoout. At MarketScout I have been leveraging statistical software and network visualization software to analyze ‘big data’. In order to hack large datasets, we need special approaches to analyse data in order to interpret it and generate meaningful insight.
The research below is a sample of some the work we have been doing for a MarketScout client.
For our analysis we focused on ASX Directors for which we have over 8,000 discrete directorships on file at just under 2,000 companies. All of this data is then combined to to build a large network of these entities. Here I define an edge between two entities as co-directorships on a board, for example if two directors sit on the same board together then an edge is defined between those two directors. This results in a graph (or social network) with 8,792 nodes and 78,426 edges. Big, but not unreasonable at all for analysis.
Next, I computed some basic network statistics on that graph. These measures are often most interesting if compared together. To highlight key directors, I generated a scatter plot of four metrics: Eigenvector centrality, betweenness centrality, closeness centrality and eccentricity.
Eigenvector centrality measures the overall centrality of a person in the network. It accounts for not only the number of connections a person has, but also the number of connections that person has to others with many connections. People with high Eigenvector centrality will be the most prominent and well-connected in a network. Alternatively, betweenness centrality measures that number of paths that go through an actor as a function of the total number of paths in the entire network. People with high betweenness will be those that act as critical bridges or cut-points between two densely connected parts of a network.
When we compare these metrics, as I have above, we can identify key directors as those that do not follow the relatively linear relationship between to two measures. Those with high betweenness but relatively low Eigenvector are central bridges within the network. What makes this comparison important, however, is that these bridges do it with few connections—hence the lower EIgenvector. Likewise, those with relatively high Eigenvector but low betweenness are network insiders. They sit inside some central region of the network, but have very few connections outside that region.
So the next step was to check closeness centrality which is represented on the plot by the colour scale. Closeness centrality measures the distance between nodes or the length of the path to others in the network essentially representing the speed at which they can access the other nodes in the work. Directors with a high closeness centrality are in an excellent position to monitor information flow – they have the best visibility into what is happening in the network. What is really interesting is seeing the clusters of closeness compared to the eigenvector and betweenness clusters.
Finally, we sized each node on the plot by eccentricity. It can be thought of as how far a director is from the director most distant from it in the graph.
