Author Robert Hoff
Updated 28 November 2018

  Projects   3D World   OpenGL   Physics   Charting   Web-work   Graphics   Uni-reports   Personal


Data collecting (web-parsing) and charting

Charts show datasets collected from the website

The charts are investement data derived from different company profiles (startups) during their campaigns on the crowdfunding platform The timelines on all charts are drawn to match the period 5 Jan 2016 to 3 Feb 2016.

The Ethos profile (a health studio - website, a brief on the project is also available - investment brief) has the most interesting activity as being the main focus of the project. The other profiles have missing intervals being added in hindsight and dependent on available data (due to their campaign start and end times).

The data was sourced by polling the html, every few minutes, from respective profiles. Here showing a representative display of such data as it appears in the browser.



The central figure is the total money raised, in this example £558,400. Together with other data points; "investors so far" and "last investment" a transcript of events can be made.


    06-Jan-2016 04:08:47  t=1.17  raised(£)=  63,610  amount(£)=   1,000  inv_c=27
    06-Jan-2016 09:26:52  t=1.39  raised(£)= 163,610  amount(£)= 100,000  inv_c=28
    06-Jan-2016 14:28:03  t=1.60  raised(£)= 163,620  amount(£)=      10  inv_c=29
    06-Jan-2016 14:41:59  t=1.61  raised(£)= 173,620  amount(£)=  10,000  inv_c=30
    06-Jan-2016 15:28:00  t=1.64  raised(£)= 188,620  amount(£)=  15,000  inv_c=31
    06-Jan-2016 15:40:58  t=1.65  raised(£)= 188,720  amount(£)=     100  inv_c=32
    06-Jan-2016 16:24:04  t=1.68  raised(£)= 188,770  amount(£)=      50  inv_c=33
    06-Jan-2016 17:43:30  t=1.74  raised(£)= 189,770  amount(£)=   1,000  inv_c=34
    06-Jan-2016 18:16:35  t=1.76  raised(£)= 190,770  amount(£)=   1,000  inv_c=35
    06-Jan-2016 19:12:49  t=1.80  raised(£)= 191,020  amount(£)=     250  inv_c=36
    06-Jan-2016 20:44:08  t=1.86  raised(£)= 191,030  amount(£)=      10  inv_c=37

Derived charts

Charts produced from the Ethos data; the line chart with investor volumes, a pie-charts of investment amounts and a breakdown of investment totals across each day.


Implemented in Java with JSoup (html parsing) and JFreeChart (charting library)

src > main >
  java > charting >
                <-- application entry, either for collecting data, parsing data or generating charts
          <-- Jframe for displaying charts
                   <-- 'T' for token, an enum holding constants of different profiles
              <-- A utility class for parsing and displaying dates
                analyse >                <-- the following classes each generating a different type of chart
                data >
                         <-- interface to the data collected, together with ParseFile and EventItem
                        <-- the Jsoup web scraper, returns the result as console output
                                                            which is appended to the data files below using the
                                                            by running a cron-job on the batch script
data >                      <-- collected data saved as text files here as "inv-data_*.txt"
        ...   <-- batch script allowing a cron-job to run the scraper at repeated intervals


© Robert Hoff