#include <DataSource.h>

Detailed Description

This class provides an API to various data sources used in BRIDGES.

BRIDGES currently supports a few external datasets for use with BRIDGES assignments: USGIS Earthquake Tweeet streaming data feed, IMDB (file), Book metadata collection, IGN Game Data, Shakespear book/poem meta data, etc.

Functions are provided that access a user specified number of data records; objects of the appropriate type are returned as a list.

Outputs more information if the environment variable FORCE_BRIDGES_DATADEBUG exists, regardless of what it is set too.

Public Member Functions
	DataSource (bridges::Bridges *br=nullptr)

	DataSource (bridges::Bridges &br)

void	setSourceType (string type)
	set data server type More...

vector< City >	getUSCities (unordered_map< string, string > params)
	Retrieves US city data based on a set of filtering parameters. More...

vector< Game >	getGameData ()
	Get meta data of the IGN games collection. More...

vector< ActorMovieIMDB >	getActorMovieIMDBData (int number=0)
	Get ActorMovie IMDB Data Data is retrieved, formatted into a list of ActorMovieIMDB objects. More...

vector< ActorMovieIMDB >	getActorMovieIMDBData2 ()
	Get ActorMovie IMDB Data Data is retrieved, formatted into a list of ActorMovieIMDB objects. More...

vector< EarthquakeUSGS >	getEarthquakeUSGSData (int number=0)
	Get USGS earthquake data USGS Tweet data (https://earthquake.usgs.gov/earthquakes/map/) retrieved, formatted into a list of EarthquakeUSGS objects. More...

vector< Shakespeare >	getShakespeareData (string type="", bool textonly=false)
	Get data of Shakespeare works (plays, poems) More...

Song	getSong (string songTitle, string artistName="")
	Get data of a particular songs (including lyrics) using the Genius API (https://docs.genius.com/), given the song title and artist name. Valid endpoints: http://bridgesdata.herokuapp.com/api/songs/find/ Valid queryParams: song title, artist name. More...

vector< Song >	getSongData ()
	Get data of the songs (including lyrics) using the Genius API https://docs.genius.com/ Valid endpoints: https://bridgesdata.herokuapp.com/api/songs/. More...

GutenbergBook	getGutenbergBookMetaData (int id=0)
	Get meta data of a single Gutenberg book This function retrieves, and formats the data into a list of GutenbergBook objects. More...

vector< GutenbergBook >	getGutenbergBookMetaData (string term, string category)
	Search the gutenberg data for retrieving meta data of books matching a string and a category. More...

string	getGutenbergBookText (int id=0)
	Get the full text of the book with the provided id. More...

vector< CancerIncidence >	getCancerIncidenceData (int num=0)
	Retrieves the CDC dataset of Cancer Incidence. Data is retrieved into a vector of records See CancerIncidence class for more information. More...

OSMData	getOSMData (double lat_min, double long_min, double lat_max, double long_max, string level="default")
	Get OpenStreetMap data given a bounding rectangle of lat/long values. More...

vector< Amenity >	getAmenityData (double minLat, double minLon, double maxLat, double maxLon, std::string amenity)

vector< Amenity >	getAmenityData (const std::string &location, const std::string &amenity)

vector< Amenity >	parseAmenityData (string amenity_json)
	Parses the amenity string and returns an AmenityData object. More...

OSMData	getOSMData (string location, string level="default")

bridges::GraphAdjList< int, std::string >	getGraphFromAssignment (const std::string &user, int assignment, int subassignment=0)

bridges::ColorGrid	getColorGridFromAssignment (const std::string &user, int assignment, int subassignment=0)

std::vector< MovieActorWikidata >	getWikidataActorMovie (int yearbegin, int yearend)
	This function returns the Movie and Actors playing in them between two years. More...

ElevationData	getElevationData (double minLat, double minLon, double maxLat, double maxLon, double res=0.0166)

ElevationData	parseElevationData (string elev_json)
	Parses the elevation data string and retuns an Elevation object. More...

std::vector< std::string >	getAvailableSubreddits ()
	retrieves the subreddits made available by BRIDGES More...

vector< Reddit >	getRedditData (string subreddit, int time_request=-9999)

Constructor & Destructor Documentation

◆ DataSource() [1/2]

bridges::DataSource::DataSource ( bridges::Bridges * br = nullptr )

inline

◆ DataSource() [2/2]

bridges::DataSource::DataSource ( bridges::Bridges & br )

inline

Member Function Documentation

◆ getActorMovieIMDBData()

vector<ActorMovieIMDB> bridges::DataSource::getActorMovieIMDBData ( int number = 0 )

inline

Get ActorMovie IMDB Data Data is retrieved, formatted into a list of ActorMovieIMDB objects.

Parameters

number the number of actor/movie pairs, but currently unused, returns all records.

Exceptions

Exception if the request fails

Returns: a list of ActorMovieIMDB objects, but only actor and movie fields in this version

◆ getActorMovieIMDBData2()

vector<ActorMovieIMDB> bridges::DataSource::getActorMovieIMDBData2 ( )

inline

Get ActorMovie IMDB Data Data is retrieved, formatted into a list of ActorMovieIMDB objects.

Exceptions

Exception if the request fails

Returns: a list of ActorMovieIMDB objects, consisting of actor name, movie name, movie genre and movie rating is returned.

◆ getAmenityData() [1/2]

vector<Amenity> bridges::DataSource::getAmenityData	(	const std::string &	location,
		const std::string &	amenity
	)

inline

This method retrieves the specified amenity related data given a location from a specified openstreet mmap location

Parameters

location	city/town from where amenity data is sought
amenity	amenity type

Returns: vector<Amenity> containing list of amenities

Exceptions

exception

◆ getAmenityData() [2/2]

vector<Amenity> bridges::DataSource::getAmenityData	(	double	minLat,
		double	minLon,
		double	maxLat,
		double	maxLon,
		std::string	amenity
	)

inline

This method retrieves the specified amenity related data given a bounding box of a region, from a Open Street map

Parameters

minLat	minimum latitude
minLon	minimumm longitude
maxLat	maximum latitude
maxLon	maximum longitude
amenity	amenity type

Returns: vector<Amenity> containing list of amenities

Exceptions

exception

◆ getAvailableSubreddits()

std::vector<std::string> bridges::DataSource::getAvailableSubreddits ( )

inline

retrieves the subreddits made available by BRIDGES

Returns: a list of strings of subreddit names

◆ getCancerIncidenceData()

vector<CancerIncidence> bridges::DataSource::getCancerIncidenceData ( int num = 0 )

inline

Retrieves the CDC dataset of Cancer Incidence. Data is retrieved into a vector of records See CancerIncidence class for more information.

◆ getColorGridFromAssignment()

bridges::ColorGrid bridges::DataSource::getColorGridFromAssignment	(	const std::string &	user,
		int	assignment,
		int	subassignment = `0`
	)

inline

Reconstruct a ColorGrid from an existing ColorGrid on the Bridges server

Returns: the ColorGrid stored in the bridges server

Parameters

user	the name of the user who uploaded the assignment
assignment	the ID of the assignment to get
subassignment	the ID of the subassignment to get

◆ getEarthquakeUSGSData()

vector<EarthquakeUSGS> bridges::DataSource::getEarthquakeUSGSData ( int number = 0 )

inline

Get USGS earthquake data USGS Tweet data (https://earthquake.usgs.gov/earthquakes/map/) retrieved, formatted into a list of EarthquakeUSGS objects.

Parameters

number the number of earthquake records retrieved, limited to 5000

Exceptions

Exception if the request fails

Returns: a list of earthquake records

◆ getElevationData()

ElevationData bridges::DataSource::getElevationData	(	double	minLat,
		double	minLon,
		double	maxLat,
		double	maxLon,
		double	res = `0.0166`
	)

inline

Returns ElevationData for the provided coordinate box at the given resolution. Note that the ElevationData that is returned may have slightly different location and resolution.

Parameters

latitMin	minimum latitude requested
longitMin	maximum latitude requested
latitMax	minimum longitude requested
longitMax	maximum longitude requested
res	spatial resolution, aka the distance between two samples (in degrees)

◆ getGameData()

vector<Game> bridges::DataSource::getGameData ( )

inline

Get meta data of the IGN games collection.

This function retrieves and formats the data into a list of Game objects

Exceptions

Exception if the request fails

Returns: a list of Game objects,

◆ getGraphFromAssignment()

bridges::GraphAdjList<int, std::string> bridges::DataSource::getGraphFromAssignment	(	const std::string &	user,
		int	assignment,
		int	subassignment = `0`
	)

inline

Reconstruct a GraphAdjList from an existing GraphAdjList on the Bridges server

The reconstructed assignment sees vertices identified as integers in the order they are stored in the server. The data associated with a vertex is a string that comes from the label of that vertices. The data associated with an edge is the string that comes from the label of that edge. The edge weights are also reobtained from the bridges server.

Parameters

user	the name of the user who uploaded the assignment
assignment	the ID of the assignment to get
subassignment	the ID of the subassignment to get

Returns: the ColorGrid stored in the bridges server

◆ getGutenbergBookMetaData() [1/2]

GutenbergBook bridges::DataSource::getGutenbergBookMetaData ( int id = 0 )

inline

Get meta data of a single Gutenberg book This function retrieves, and formats the data into a list of GutenbergBook objects.

Parameters

id	Gutenberg book id

Returns: metadata of the book

◆ getGutenbergBookMetaData() [2/2]

vector<GutenbergBook> bridges::DataSource::getGutenbergBookMetaData	(	string	term,
		string	category
	)

inline

Search the gutenberg data for retrieving meta data of books matching a string and a category.

Data is retrieved into a vector of book records

Parameters

term	a string that matches the category
category	category can be any book attribute (title, genre, date, Library of Congress class, language)

◆ getGutenbergBookText()

string bridges::DataSource::getGutenbergBookText ( int id = 0 )

inline

Get the full text of the book with the provided id.

Data is retrieved into a text string

Parameters

id	id of Gutenberg book

◆ getOSMData() [1/2]

OSMData bridges::DataSource::getOSMData	(	double	lat_min,
		double	long_min,
		double	lat_max,
		double	long_max,
		string	level = `"default"`
	)

inline

Get OpenStreetMap data given a bounding rectangle of lat/long values.

The function also take a level of detail which can be anything in ["motorway", "trunk", "primary", "secondary", "tertiary, "unclassified", "residential", "living_street", "service", "trails", "walking", "bicycle" ]

Parameters

lat_min	latitude minimum
long_min	longitude minimum
lat_max	latitude maximum
long_max	longitude maximum
level	data resolution

Exceptions

Exception if the request fails

Returns: an OSMData object

◆ getOSMData() [2/2]

OSMData bridges::DataSource::getOSMData	(	string	location,
		string	level = `"default"`
	)

inline

Get OpenStreetMap data given a city name and resolution level

The function also take a level of detail which can be anything in ["motorway", "trunk", "primary", "secondary", "tertiary, "unclassified", "residential", "living_street", "service", "trails", "walking", "bicycle" ]

Parameters

location	location name (string)
level	data resolution

Exceptions

Exception if the request fails

Returns: an OSMData object

◆ getRedditData()

vector<Reddit> bridges::DataSource::getRedditData	(	string	subreddit,
		int	time_request = `-9999`
	)

inline

@brief retrieves the reddit posts from a subreddit

Parameters

subreddit	the name of the subreddit ( check list available at http://bridges-data-server-reddit.bridgesuncc.org/list or using getAvailableSubreddits() )
time_request	unix timestamp of when requested subreddit was generated or less than 0 for now

Returns: a list of reddit objects with the data of the posts

◆ getShakespeareData()

vector<Shakespeare> bridges::DataSource::getShakespeareData	(	string	type = `""`,
		bool	textonly = `false`
	)

inline

Get data of Shakespeare works (plays, poems)

This function retrieves and formats the data into a a list of Shakespeare objects.

Exceptions

Exception if the request fails

Parameters

type	can be "plays", "poems", or "". If this is specified, then only these types of works are retrieved.
textonly	if this is set, then only the text is retrieved. (that is to say punctuation is stripped out)

Returns: an array of Shakespeare objects

◆ getSong()

Song bridges::DataSource::getSong	(	string	songTitle,
		string	artistName = `""`
	)

inline

Get data of a particular songs (including lyrics) using the Genius API (https://docs.genius.com/), given the song title and artist name. Valid endpoints: http://bridgesdata.herokuapp.com/api/songs/find/ Valid queryParams: song title, artist name.

This function retrieves and formats the data into a Song object. The song if not cached in the local DB is queried and added to the DB

Parameters

songTitle	title of the song. inexact is ok, will be matched by genisu
artistName	name of artist. empty string if unspecified. If specified, must be exact

Exceptions

Exception if the request fails

Returns: a Song object,

◆ getSongData()

vector<Song> bridges::DataSource::getSongData ( )

inline

Get data of the songs (including lyrics) using the Genius API https://docs.genius.com/ Valid endpoints: https://bridgesdata.herokuapp.com/api/songs/.

This function retrieves and formats the data into a list of Song objects. This version of the API retrieves all the cached songs in the local DB.

Exceptions

Exception if the request fails

Returns: a list of Song objects,

◆ getUSCities()

vector<City> bridges::DataSource::getUSCities ( unordered_map< string, string > params )

inline

Retrieves US city data based on a set of filtering parameters.

Parameters

params this represents a specification of the filtering parameters provided as a map. Multiple parameters will result in filtering as a combination (intersection) Available parameters and their types are as follows: 'city' : string 'state' : string 'country' : string 'time_zone' : string 'min_elev' : integer 'max_elev' : integer 'min_pop' : integer 'max_pop' : integer 'min_lat' : float – Lat minimum 'max_lat' : float – Lat maximum 'min_long' : float – Lat minimum 'max_long' : float – Lat maximum 'limit' : integer – max number of cities to return

◆ getWikidataActorMovie()

std::vector<MovieActorWikidata> bridges::DataSource::getWikidataActorMovie	(	int	yearbegin,
		int	yearend
	)

inline

This function returns the Movie and Actors playing in them between two years.

Return movie pair in the [yearbegin; yearend] interval.

Parameters

yearbegin	first year to include
yearend	last year to include

◆ parseAmenityData()

vector<Amenity> bridges::DataSource::parseAmenityData ( string amenity_json )

inline

Parses the amenity string and returns an AmenityData object.

Parameters

amenity_json string of the url that will be used when requesting amenity data from server

Returns: vector<Amenity> containing list of amenities

Exceptions

If	there is an error parsing response from server or is an invalid location name

◆ parseElevationData()

ElevationData bridges::DataSource::parseElevationData ( string elev_json )

inline

Parses the elevation data string and retuns an Elevation object.

Parameters

elev_json string containing the requested elevation data

Returns: Elevation data object

◆ setSourceType()

void bridges::DataSource::setSourceType ( string type )

inline

set data server type

Parameters

string data server type, can be 'live', 'testing', or 'local')

Exceptions

exception if incorrect type is passed

The documentation for this class was generated from the following file:

/home/erik/work/bridges/bridges-cxx/src/DataSource.h

Detailed Description

Public Member Functions

Constructor & Destructor Documentation

◆ DataSource() [1/2]

◆ DataSource() [2/2]

Member Function Documentation

◆ getActorMovieIMDBData()

◆ getActorMovieIMDBData2()

◆ getAmenityData() [1/2]

◆ getAmenityData() [2/2]

◆ getAvailableSubreddits()

◆ getCancerIncidenceData()

◆ getColorGridFromAssignment()

◆ getEarthquakeUSGSData()

◆ getElevationData()

◆ getGameData()

◆ getGraphFromAssignment()

◆ getGutenbergBookMetaData() [1/2]

◆ getGutenbergBookMetaData() [2/2]

◆ getGutenbergBookText()

◆ getOSMData() [1/2]

◆ getOSMData() [2/2]

◆ getRedditData()

◆ getShakespeareData()

◆ getSong()

◆ getSongData()

◆ getUSCities()

◆ getWikidataActorMovie()

◆ parseAmenityData()

◆ parseElevationData()

◆ setSourceType()