2018 Project 2

Learning to Fly

Team member choice due 2/16/18 at 8:59 pm Chicago time
Project alpha version due 3/5/18 at 8:59 pm Chicago time
Project final version due 3/19/18 at
8:59 pm Chicago time

Project 2 will be the first group project and will focus on interactive visualization about airline flights in Illinois. While project 1 showed a variety of pretty much static visualizations, project 2 will focus on more interactive visualizations that can be used to investigate a dataset.

The standard group size will be 3 or 4 people per group. You can chose who you want to be in a group with. I will create groups on Sunday 2/18 for people that have not formed groups on their own, but you must email me that you are continuing in the class and want to be placed into a team - I do not want to create teams with people that are not going to be serious about working with their partners. Be sure to take a look at the project 1 web pages to get a better idea of people's abilities as you form your teams. Once you have a team please send andy an email CC'd to all the team members confirming that you are a group.

You should very quickly set up a web page for your group project and send the URL to andy. The final webpage for the project will be public; the in-process web pages do not need to be public as long as andy and the TA have access. Each Friday of the project each team member should post on the project web site an overview of what he/she did on the project that week. This comes in handy when assigning ratings to your collaborators and making sure that everyone is contributing in a timely fashion.



The data for this project is available from
https://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=236&DB_Short_Name=On-Time

You may also find this script helpful:
https://gist.github.com/bbengfort/7322184

We are going to focus on Illinois (filter geography on the site) in 2017, so the data is roughly is 6MB per month.

The data to focus on includes:
FlightDate

Carrier
FlightNum

OriginAirportID
OriginCityName

DestAirportID
DestCityName

DepTime (local time)

ArrTime (local time)

Cancelled
Cancellation Code

AirTime
Distance

5 causes of delay

Note that there are quite a few Illinois and Chicago airports - there is a lookup table on the same website to disambiguate the airport IDs

As of 'now' there is data available through November 2017. If all goes well the data for December should also be available before the project is due.




For a C you need to create a useful, responsive interactive visualization for 1 month focusing on O'Hare and Midway


For a B you need to add:


For an A you need to add:


Graduate Students need to add:

In all of these case you need to make sure that your visualizations are well constructed with good color and font choices, proper labeling, and that they effectively reveal the truth about the data to the user

Note that as part of the web page part of the grade you will need to use your interface to show your findings, so make sure that the way your interface displays information is clear.

As with all the projects your app will be evaluated running full screen on the classroom wall and should not require scrolling


For this project you should again host your solution using Shinyapps.io or through the shiny.evl.uic.edu site



There are two deadlines for this project. By the first deadline you should have implemented the initial screen layout of your application and have the basic functionality allowing the user to perform an example of the various 'C' functionality. This will make sure that your group is on track and that you can focus on making a good interface and set of visualizations, not just functional ones. Personally, I think you should have the entire C functionality done at that point if you are going for an A on the project as a whole. You should make a link to this version of the interface available on your group project page.

As part of the final turn in you should create a set of web pages that describe your work on the project. This should include:

all of which should have plenty of screenshots with meaningful captions. Web pages like this can be very helpful later on in helping you build up a portfolio of your work when you start looking for a job so please put some effort into it.

Be sure to document any external libraries or tools that you make use of - give credit where credit is due.

You should also create a 2-3 minute YouTube video showing the use of your application including narration with decent audio quality. That video should be in a very obvious place on your main project web page. The easiest way to do this is to use a screen-capture tool while interacting with your application, though you will most likely find its useful to do some editing afterwards to tighten the video up. Its also a good idea to have a video like this available as a backup during your presentation just in case of gremlins.
You may want to shoot this video on the wall itself.

The web page including screen snapshots and video need to be done by the deadline so be sure to leave enough time to get that work done.

I will be linking your web page to the course notes so please send me a nice jpg image of your visualization for the web. This should be named p2.<someone_in_your_groups_last_name>.jpg. 



When the project is done, each person in the group should also send Andy a private email with no one else CC'd ranking your coworkers on the project on a scale from 1 (low) to 5 (high) in terms of how good a coworker they were on the project. If you never want to work with them again, give them a 1. If this person would be a first choice for a partner on a future project then give them a 5. If they did what was expected but nothing particularly good or bad then give them a 3. By default your score should be 3 unless you have a particular reason to increase or decrease the number. If you are giving a score other than 3 you need to say why. Please confine your responses to 1, 2, 3, 4, 5 and no 1/3ds or .5s please. Each person's score on the project will be based on the overall score for the group modified by these rankings.



Each group will show their visualization to the class and describe its features. This allows everyone to see a variety of solutions to the problem, and a variety of implementations. Rehearse your presentation ... several times. All team members are expected to participate equally in that presentation.



Has a group:
1. Sohal, Barapatre, Revanna Shivaprabhu, Phaltankar
2. Sinha, Katwala, Thmmu
3. Flower, Nishimoto, Guo, Azhari
4. Pham, Galante, Waller
5. Mahida, Kupiec, Lindmae
6. Borges, Hauck, Khan, Chandarana
7. Rojas Hernandez, Rice, Hao, Basu
8. Struck, Stranczek, Janowski
9. Hanula, Kirilov, Vig
10. Arcivar, Samad, Velazquez, Biegaj, Al-Khatib



There was some interest in class in being able to use plotly on the project. It has are some nice capabilities but if you rely on the basic controls you can create some really bad / unusable visualizations, so you are going to want to have your own controls to make sure the user can easily see what they need to see and not just provide controls that do things for the sake of doing them, because its very easy to overwhelm the user with buttons and in a few clicks make the visualizations unreadable. Engineers really love giving users increased capabilities without thinking enough about how doing that can make people's lives more difficult. So ... using plotly is OK on the assignment, but be aware that it can be a trap.

There is now a virtual machine set up in evl to serve shiny/R applications.

This should show my evl weather app running in evl - http://shiny.evl.uic.edu:3838/evlweather

you should be able to ssh into shiny.evl.uic.edu as user gX where X is your group number. We will talk about the passwords in class. Please change the password ASAP.

and then create a directory as /srv/shiny-server/gX/app_name
and then place your files in the app_name directory

and then you should be able to see the app running as http://shiny.evl.uic.edu:3838/gX/app_name


as of 3/16/18 at 11am these are the installed packages on shiny.evl.uic.edu

installed.packages()[,1:2]
               Package          LibPath                       
assertthat     "assertthat"     "/usr/local/lib/R/site-library"
backports      "backports"      "/usr/local/lib/R/site-library"
base64enc      "base64enc"      "/usr/local/lib/R/site-library"
BH             "BH"             "/usr/local/lib/R/site-library"
bindr          "bindr"          "/usr/local/lib/R/site-library"
bindrcpp       "bindrcpp"       "/usr/local/lib/R/site-library"
cli            "cli"            "/usr/local/lib/R/site-library"
colorspace     "colorspace"     "/usr/local/lib/R/site-library"
crayon         "crayon"         "/usr/local/lib/R/site-library"
crosstalk      "crosstalk"      "/usr/local/lib/R/site-library"
curl           "curl"           "/usr/local/lib/R/site-library"
data.table     "data.table"     "/usr/local/lib/R/site-library"
devtools       "devtools"       "/usr/local/lib/R/site-library"
dichromat      "dichromat"      "/usr/local/lib/R/site-library"
digest         "digest"         "/usr/local/lib/R/site-library"
dplyr          "dplyr"          "/usr/local/lib/R/site-library"
DT             "DT"             "/usr/local/lib/R/site-library"
dygraphs       "dygraphs"       "/usr/local/lib/R/site-library"
evaluate       "evaluate"       "/usr/local/lib/R/site-library"
fasttime       "fasttime"       "/usr/local/lib/R/site-library"
geojson        "geojson"        "/usr/local/lib/R/site-library"
geojsonio      "geojsonio"      "/usr/local/lib/R/site-library"
ggplot2        "ggplot2"        "/usr/local/lib/R/site-library"
ggridges       "ggridges"       "/usr/local/lib/R/site-library"
ggthemes       "ggthemes"       "/usr/local/lib/R/site-library"
git2r          "git2r"          "/usr/local/lib/R/site-library"
glue           "glue"           "/usr/local/lib/R/site-library"
gridExtra      "gridExtra"      "/usr/local/lib/R/site-library"
gtable         "gtable"         "/usr/local/lib/R/site-library"
hexbin         "hexbin"         "/usr/local/lib/R/site-library"
highr          "highr"          "/usr/local/lib/R/site-library"
hms            "hms"            "/usr/local/lib/R/site-library"
htmltools      "htmltools"      "/usr/local/lib/R/site-library"
htmlwidgets    "htmlwidgets"    "/usr/local/lib/R/site-library"
httpuv         "httpuv"         "/usr/local/lib/R/site-library"
httr           "httr"           "/usr/local/lib/R/site-library"
jpeg           "jpeg"           "/usr/local/lib/R/site-library"
jqr            "jqr"            "/usr/local/lib/R/site-library"
jsonlite       "jsonlite"       "/usr/local/lib/R/site-library"
knitr          "knitr"          "/usr/local/lib/R/site-library"
labeling       "labeling"       "/usr/local/lib/R/site-library"
lazyeval       "lazyeval"       "/usr/local/lib/R/site-library"
leaflet        "leaflet"        "/usr/local/lib/R/site-library"
lubridate      "lubridate"      "/usr/local/lib/R/site-library"
magrittr       "magrittr"       "/usr/local/lib/R/site-library"
maptools       "maptools"       "/usr/local/lib/R/site-library"
markdown       "markdown"       "/usr/local/lib/R/site-library"
memoise        "memoise"        "/usr/local/lib/R/site-library"
mime           "mime"           "/usr/local/lib/R/site-library"
munsell        "munsell"        "/usr/local/lib/R/site-library"
openssl        "openssl"        "/usr/local/lib/R/site-library"
pillar         "pillar"         "/usr/local/lib/R/site-library"
pkgconfig      "pkgconfig"      "/usr/local/lib/R/site-library"
plogr          "plogr"          "/usr/local/lib/R/site-library"
plotly         "plotly"         "/usr/local/lib/R/site-library"
plyr           "plyr"           "/usr/local/lib/R/site-library"
png            "png"            "/usr/local/lib/R/site-library"
protolite      "protolite"      "/usr/local/lib/R/site-library"
purrr          "purrr"          "/usr/local/lib/R/site-library"
R6             "R6"             "/usr/local/lib/R/site-library"
raster         "raster"         "/usr/local/lib/R/site-library"
RColorBrewer   "RColorBrewer"   "/usr/local/lib/R/site-library"
Rcpp           "Rcpp"           "/usr/local/lib/R/site-library"
readr          "readr"          "/usr/local/lib/R/site-library"
reshape2       "reshape2"       "/usr/local/lib/R/site-library"
rgdal          "rgdal"          "/usr/local/lib/R/site-library"
rgeos          "rgeos"          "/usr/local/lib/R/site-library"
rlang          "rlang"          "/usr/local/lib/R/site-library"
rmarkdown      "rmarkdown"      "/usr/local/lib/R/site-library"
rprojroot      "rprojroot"      "/usr/local/lib/R/site-library"
rstudioapi     "rstudioapi"     "/usr/local/lib/R/site-library"
scales         "scales"         "/usr/local/lib/R/site-library"
shiny          "shiny"          "/usr/local/lib/R/site-library"
shinyBS        "shinyBS"        "/usr/local/lib/R/site-library"
shinydashboard "shinydashboard" "/usr/local/lib/R/site-library"
shinyjs        "shinyjs"        "/usr/local/lib/R/site-library"
shinythemes    "shinythemes"    "/usr/local/lib/R/site-library"
sourcetools    "sourcetools"    "/usr/local/lib/R/site-library"
sp             "sp"             "/usr/local/lib/R/site-library"
streamgraph    "streamgraph"    "/usr/local/lib/R/site-library"
stringi        "stringi"        "/usr/local/lib/R/site-library"
stringr        "stringr"        "/usr/local/lib/R/site-library"
tibble         "tibble"         "/usr/local/lib/R/site-library"
tidyr          "tidyr"          "/usr/local/lib/R/site-library"
tidyselect     "tidyselect"     "/usr/local/lib/R/site-library"
txtplot        "txtplot"        "/usr/local/lib/R/site-library"
utf8           "utf8"           "/usr/local/lib/R/site-library"
V8             "V8"             "/usr/local/lib/R/site-library"
viridis        "viridis"        "/usr/local/lib/R/site-library"
viridisLite    "viridisLite"    "/usr/local/lib/R/site-library"
whisker        "whisker"        "/usr/local/lib/R/site-library"
withr          "withr"          "/usr/local/lib/R/site-library"
xtable         "xtable"         "/usr/local/lib/R/site-library"
xts            "xts"            "/usr/local/lib/R/site-library"
yaml           "yaml"           "/usr/local/lib/R/site-library"
zoo            "zoo"            "/usr/local/lib/R/site-library"
base           "base"           "/usr/lib/R/library"          
boot           "boot"           "/usr/lib/R/library"          
class          "class"          "/usr/lib/R/library"          
cluster        "cluster"        "/usr/lib/R/library"          
codetools      "codetools"      "/usr/lib/R/library"          
compiler       "compiler"       "/usr/lib/R/library"          
datasets       "datasets"       "/usr/lib/R/library"          
foreign        "foreign"        "/usr/lib/R/library"          
graphics       "graphics"       "/usr/lib/R/library"          
grDevices      "grDevices"      "/usr/lib/R/library"          
grid           "grid"           "/usr/lib/R/library"          
KernSmooth     "KernSmooth"     "/usr/lib/R/library"          
lattice        "lattice"        "/usr/lib/R/library"          
MASS           "MASS"           "/usr/lib/R/library"          
Matrix         "Matrix"         "/usr/lib/R/library"          
methods        "methods"        "/usr/lib/R/library"          
mgcv           "mgcv"           "/usr/lib/R/library"          
nlme           "nlme"           "/usr/lib/R/library"          
nnet           "nnet"           "/usr/lib/R/library"          
parallel       "parallel"       "/usr/lib/R/library"          
rpart          "rpart"          "/usr/lib/R/library"          
spatial        "spatial"        "/usr/lib/R/library"          
splines        "splines"        "/usr/lib/R/library"          
stats          "stats"          "/usr/lib/R/library"          
stats4         "stats4"         "/usr/lib/R/library"          
survival       "survival"       "/usr/lib/R/library"          
tcltk          "tcltk"          "/usr/lib/R/library"          
tools          "tools"          "/usr/lib/R/library"          
utils          "utils"          "/usr/lib/R/library"          



last revision  3/16/18 - added list of installed packages on evl shiny server
3/6/18 text in A range more obvious about allowing the user to pick ...
3/1/18 more specificity about the meaning of 'all domestic airlines' in the C and B ranges
2/22/18 - updated text on number of airlines to show to
all of the domestic ones, as that is a smaller number than in the original assignment.
2/20/18 - updating groups, added note about plotly, and evl shiny server