2019 Project 2

Every Breath You Take

Team member choice due 2/16/19 at 8:59 pm Chicago time
Project alpha version due 3/4/19 at 8:59 pm Chicago time
Project final version due 3/18/19 at
8:59 pm Chicago time

Project 2 will be the first group project and will take a deeper look at the air quality data used in Project 1. In project 1 we looked at the annual data. Here we are going to look at the daily and hourly data. This lets us investigate whether there is more pollution when it�s warmer, during the daytime / work hours? during the work week or on weekends? when its windy? in the north / south, near the coast or not, near more rural, urban, or industrial areas, etc.

Again we will be using the main classroom wall as the target display device, but this time with touch enabled, so make sure all of your controls are reachable by a reasonably sized human without needing a step ladder.

The standard group size will be 2 or 3 people per group. You can chose who you want to be in a group with. I will create groups on Sunday 2/17 for people that have not formed groups on their own, but you must email me that you are continuing in the class and want to be placed into a team. I do not want to create teams with people that are not going to be serious about working with their partners. Be sure to take a look at the project 1 web pages to get a better idea of people's abilities as you form your teams. Once you have a team, or part of a team,  please send andy an email CC'd to all the team members confirming that you are a group.

You should very quickly set up a web page for your group project and send the URL to andy. The final webpage for the project will be public; the in-process web pages do not need to be public as long as andy and the TA have access. Each Friday of the project each team member should post on the project web site an overview of what he/she did on the project that week. This comes in handy when assigning ratings to your collaborators and making sure that everyone is contributing in a timely fashion.

You can use the version of Project 1 from one of your team members as a starting point if you want, or start over.

As there will be larger file sizes this time you may want to do some pre-processing on the files. This is ok as long as it is an automated process that can be re-run to convert the downloaded datafiles into the versions your application uses, and in particular andy and the TA need to be able to easy re-run this pre-processing.



The data for this project is available from the same place as project 1: https://aqs.epa.gov/aqsweb/airdata/download_files.html




For a C you need to


For a B you need to add:


For an A you need to add:


Graduate Students need to add:

In all of these case you need to make sure that your visualizations are well constructed with good color and font choices, proper labeling, and that they effectively reveal the truth about the data to the user

Note that as part of the web page part of the grade you will need to use your interface to show your findings, so make sure that the way your interface displays information is clear.

As with all the projects your app will be evaluated running full screen on the classroom wall and should not require scrolling


For this project we will host your solutions through the shiny.evl.uic.edu site

I recommend using github or bitcucket or a similar service to deal with your project management - it makes it easy to see who did what when, easy to roll back, keeps copies of the code safe, etc.



There are two deadlines for this project. By the first deadline you should have implemented the initial screen layout of your application and have the basic functionality allowing the user to perform an example of the various 'C' functionality. This will make sure that your group is on track and that you can focus on making a good interface and set of visualizations, not just functional ones. Personally, I think you should have the entire C functionality done at that point if you are going for an A on the project as a whole. You should make a link to this version of the interface available on your group project page.

As part of the final turn in you should create a set of web pages that describe your work on the project. This should include:

all of which should have plenty of screenshots with meaningful captions. Web pages like this can be very helpful later on in helping you build up a portfolio of your work when you start looking for a job so please put some effort into it.

Be sure to document any external libraries, tools, etc. that you make use of - give credit where credit is due for everything that you didn't create yourself.

You should also create a 2-3 minute YouTube video showing the use of your application including narration with decent audio quality. That video should be in a very obvious place on your main project web page. The easiest way to do this is to use a screen-capture tool while interacting with your application, though you will most likely find its useful to do some editing afterwards to tighten the video up. Its also a good idea to have a video like this available as a backup during your presentation just in case of gremlins.
You may want to shoot this video on the wall itself.

The web page including screen snapshots and video need to be done by the deadline so be sure to leave enough time to get that work done.

I will be linking your web page to the course notes so please send me a nice jpg image of your visualization for the web. This should be named p2.<someone_in_your_groups_last_name>.jpg. 



When the project is done, each person in the group should also send Andy a private email with no one else CC'd ranking your coworkers on the project on a scale from 1 (low) to 5 (high) in terms of how good a coworker they were on the project. If you never want to work with them again, give them a 1. If this person would be a first choice for a partner on a future project then give them a 5. If they did what was expected but nothing particularly good or bad then give them a 3. By default your score should be 3 unless you have a particular reason to increase or decrease the number. If you are giving a score other than 3 you need to say why. Please confine your responses to 1, 2, 3, 4, 5 and no 1/3ds or .5s please. Each person's score on the project will be based on the overall score for the group modified by these rankings.



Each group will show their visualization to the class and describe its features. This allows everyone to see a variety of solutions to the problem, and a variety of implementations. Rehearse your presentation ... several times. All team members are expected to participate equally in that presentation.



Has a group:

1. Masoud, Rami & Munoz, Jonathan & Kumar, Mudit & Raol, Jeet
2. Laczny, Nicole & Ptasznik, Conrad & Charlesworth, Casey-Leigh & Abdul salam, Mohammad
3. Thiruvarpu Neelakantan, Sai Krishnan & Chandrasekaran, Praveen & Jayaraman, Varsha & Aleem, Abdullah
4. Mantovani, Mirko & Vasudevan, Abhishek & Khemani, Ashwani
5. Toher, William & Vo, Dylan & Bedu, Wilfried Christophe
6. Qarni, Fatima & Martinez, Emmanuel
7. Patel, Bhargav & Ilic, Aleksandar & Brahmbhatt, Rohan & Lin, Catherine
8. Baiju, Aiswarya & Baig, Mirza & Badriyah, Arsh
9. Nunez, Jhon & Arellano, Daisy
10. Kumar, Vikas & Rajesh, Yazhini
11. Lambert, Stephen & Patel, Jay
12. Lazcano, Cesar & Patel, Kevalkumar
13. Tlemat, Anas & Hatten, Danielle & Hruby, Raymond




There was some interest in class in being able to use plotly on the project. It has are some nice capabilities but if you rely on the basic controls you can create some really bad / unusable visualizations, so you are going to want to have your own controls to make sure the user can easily see what they need to see and not just provide controls that do things for the sake of doing them, because its very easy to overwhelm the user with buttons and in a few clicks make the visualizations unreadable. Engineers really love giving users increased capabilities without thinking enough about how doing that can make people's lives more difficult. So ... using plotly is OK on the assignment, but be aware that it can be a trap.

There is now a virtual machine set up in evl to serve shiny/R applications.

This should show my evl weather app running in evl - http://shiny.evl.uic.edu:3838/evlweather

you should be able to ssh into shiny.evl.uic.edu as user gX where X is your group number. We will talk about the passwords in class. Please change the password ASAP.

and then you will find the directory /srv/shiny-server/gX/app_name
where you can place your files in the app_name directory

and then you should be able to see the app running as http://shiny.evl.uic.edu:3838/gX/app_name


as of 3/4/19 these are the installed packages on shiny.evl.uic.edu

 

> installed.packages()[,1:2]

                Package           LibPath                        

askpass         "askpass"         "/usr/local/lib/R/site-library"

assertthat      "assertthat"      "/usr/local/lib/R/site-library"

backports       "backports"       "/usr/local/lib/R/site-library"

base64enc       "base64enc"       "/usr/local/lib/R/site-library"

BH              "BH"              "/usr/local/lib/R/site-library"

bindr           "bindr"           "/usr/local/lib/R/site-library"

bindrcpp        "bindrcpp"        "/usr/local/lib/R/site-library"

brew            "brew"            "/usr/local/lib/R/site-library"

broom           "broom"           "/usr/local/lib/R/site-library"

callr           "callr"           "/usr/local/lib/R/site-library"

cdlTools        "cdlTools"        "/usr/local/lib/R/site-library"

cellranger      "cellranger"      "/usr/local/lib/R/site-library"

classInt        "classInt"        "/usr/local/lib/R/site-library"

cli             "cli"             "/usr/local/lib/R/site-library"

clipr           "clipr"           "/usr/local/lib/R/site-library"

clisymbols      "clisymbols"      "/usr/local/lib/R/site-library"

colorspace      "colorspace"      "/usr/local/lib/R/site-library"

colourpicker    "colourpicker"    "/usr/local/lib/R/site-library"

crayon          "crayon"          "/usr/local/lib/R/site-library"

crosstalk       "crosstalk"       "/usr/local/lib/R/site-library"

curl            "curl"            "/usr/local/lib/R/site-library"

dashboardthemes "dashboardthemes" "/usr/local/lib/R/site-library"

data.table      "data.table"      "/usr/local/lib/R/site-library"

DBI             "DBI"             "/usr/local/lib/R/site-library"

dbplyr          "dbplyr"          "/usr/local/lib/R/site-library"

desc            "desc"            "/usr/local/lib/R/site-library"

devtools        "devtools"        "/usr/local/lib/R/site-library"

dichromat       "dichromat"       "/usr/local/lib/R/site-library"

digest          "digest"          "/usr/local/lib/R/site-library"

dplyr           "dplyr"           "/usr/local/lib/R/site-library"

DT              "DT"              "/usr/local/lib/R/site-library"

dygraphs        "dygraphs"        "/usr/local/lib/R/site-library"

e1071           "e1071"           "/usr/local/lib/R/site-library"

ellipsis        "ellipsis"        "/usr/local/lib/R/site-library"

evaluate        "evaluate"        "/usr/local/lib/R/site-library"

fansi           "fansi"           "/usr/local/lib/R/site-library"

fasttime        "fasttime"        "/usr/local/lib/R/site-library"

feather         "feather"         "/usr/local/lib/R/site-library"

forcats         "forcats"         "/usr/local/lib/R/site-library"

fs              "fs"              "/usr/local/lib/R/site-library"

fst             "fst"             "/usr/local/lib/R/site-library"

gdtools         "gdtools"         "/usr/local/lib/R/site-library"

generics        "generics"        "/usr/local/lib/R/site-library"

geojson         "geojson"         "/usr/local/lib/R/site-library"

geojsonio       "geojsonio"       "/usr/local/lib/R/site-library"

ggplot2         "ggplot2"         "/usr/local/lib/R/site-library"

ggridges        "ggridges"        "/usr/local/lib/R/site-library"

ggthemes        "ggthemes"        "/usr/local/lib/R/site-library"

gh              "gh"              "/usr/local/lib/R/site-library"

git2r           "git2r"           "/usr/local/lib/R/site-library"

glue            "glue"            "/usr/local/lib/R/site-library"

gridExtra       "gridExtra"       "/usr/local/lib/R/site-library"

gtable          "gtable"          "/usr/local/lib/R/site-library"

haven           "haven"           "/usr/local/lib/R/site-library"

hexbin          "hexbin"          "/usr/local/lib/R/site-library"

highr           "highr"           "/usr/local/lib/R/site-library"

hms             "hms"             "/usr/local/lib/R/site-library"

htmltools       "htmltools"       "/usr/local/lib/R/site-library"

htmlwidgets     "htmlwidgets"     "/usr/local/lib/R/site-library"

httpuv          "httpuv"          "/usr/local/lib/R/site-library"

httr            "httr"            "/usr/local/lib/R/site-library"

ini             "ini"             "/usr/local/lib/R/site-library"

jpeg            "jpeg"            "/usr/local/lib/R/site-library"

jqr             "jqr"             "/usr/local/lib/R/site-library"

jsonlite        "jsonlite"        "/usr/local/lib/R/site-library"

knitr           "knitr"           "/usr/local/lib/R/site-library"

labeling        "labeling"        "/usr/local/lib/R/site-library"

later           "later"           "/usr/local/lib/R/site-library"

lazyeval        "lazyeval"        "/usr/local/lib/R/site-library"

leaflet         "leaflet"         "/usr/local/lib/R/site-library"

lubridate       "lubridate"       "/usr/local/lib/R/site-library"

lwgeom          "lwgeom"          "/usr/local/lib/R/site-library"

magrittr        "magrittr"        "/usr/local/lib/R/site-library"

maptools        "maptools"        "/usr/local/lib/R/site-library"

mapview         "mapview"         "/usr/local/lib/R/site-library"

markdown        "markdown"        "/usr/local/lib/R/site-library"

memoise         "memoise"         "/usr/local/lib/R/site-library"

mime            "mime"            "/usr/local/lib/R/site-library"

miniUI          "miniUI"          "/usr/local/lib/R/site-library"

modelr          "modelr"          "/usr/local/lib/R/site-library"

munsell         "munsell"         "/usr/local/lib/R/site-library"

NLP             "NLP"             "/usr/local/lib/R/site-library"

openssl         "openssl"         "/usr/local/lib/R/site-library"

pillar          "pillar"          "/usr/local/lib/R/site-library"

pkgbuild        "pkgbuild"        "/usr/local/lib/R/site-library"

pkgconfig       "pkgconfig"       "/usr/local/lib/R/site-library"

pkgload         "pkgload"         "/usr/local/lib/R/site-library"

plogr           "plogr"           "/usr/local/lib/R/site-library"

plotly          "plotly"          "/usr/local/lib/R/site-library"

plyr            "plyr"            "/usr/local/lib/R/site-library"

png             "png"             "/usr/local/lib/R/site-library"

prettyunits     "prettyunits"     "/usr/local/lib/R/site-library"

processx        "processx"        "/usr/local/lib/R/site-library"

progress        "progress"        "/usr/local/lib/R/site-library"

promises        "promises"        "/usr/local/lib/R/site-library"

protolite       "protolite"       "/usr/local/lib/R/site-library"

ps              "ps"              "/usr/local/lib/R/site-library"

purrr           "purrr"           "/usr/local/lib/R/site-library"

R6              "R6"              "/usr/local/lib/R/site-library"

raster          "raster"          "/usr/local/lib/R/site-library"

rcmdcheck       "rcmdcheck"       "/usr/local/lib/R/site-library"

RColorBrewer    "RColorBrewer"    "/usr/local/lib/R/site-library"

Rcpp            "Rcpp"            "/usr/local/lib/R/site-library"

readr           "readr"           "/usr/local/lib/R/site-library"

readxl          "readxl"          "/usr/local/lib/R/site-library"

rematch         "rematch"         "/usr/local/lib/R/site-library"

remotes         "remotes"         "/usr/local/lib/R/site-library"

reprex          "reprex"          "/usr/local/lib/R/site-library"

reshape2        "reshape2"        "/usr/local/lib/R/site-library"

rgdal           "rgdal"           "/usr/local/lib/R/site-library"

rgeos           "rgeos"           "/usr/local/lib/R/site-library"

rlang           "rlang"           "/usr/local/lib/R/site-library"

rmarkdown       "rmarkdown"       "/usr/local/lib/R/site-library"

rprojroot       "rprojroot"       "/usr/local/lib/R/site-library"

rstudioapi      "rstudioapi"      "/usr/local/lib/R/site-library"

rvest           "rvest"           "/usr/local/lib/R/site-library"

satellite       "satellite"       "/usr/local/lib/R/site-library"

scales          "scales"          "/usr/local/lib/R/site-library"

selectr         "selectr"         "/usr/local/lib/R/site-library"

sessioninfo     "sessioninfo"     "/usr/local/lib/R/site-library"

sf              "sf"              "/usr/local/lib/R/site-library"

shiny           "shiny"           "/usr/local/lib/R/site-library"

shinyalert      "shinyalert"      "/usr/local/lib/R/site-library"

shinyBS         "shinyBS"         "/usr/local/lib/R/site-library"

shinycssloaders "shinycssloaders" "/usr/local/lib/R/site-library"

shinydashboard  "shinydashboard"  "/usr/local/lib/R/site-library"

shinyjs         "shinyjs"         "/usr/local/lib/R/site-library"

shinythemes     "shinythemes"     "/usr/local/lib/R/site-library"

shinyWidgets    "shinyWidgets"    "/usr/local/lib/R/site-library"

slam            "slam"            "/usr/local/lib/R/site-library"

sourcetools     "sourcetools"     "/usr/local/lib/R/site-library"

sp              "sp"              "/usr/local/lib/R/site-library"

splitstackshape "splitstackshape" "/usr/local/lib/R/site-library"

stringi         "stringi"         "/usr/local/lib/R/site-library"

stringr         "stringr"         "/usr/local/lib/R/site-library"

svglite         "svglite"         "/usr/local/lib/R/site-library"

sys             "sys"             "/usr/local/lib/R/site-library"

tibble          "tibble"          "/usr/local/lib/R/site-library"

tidyr           "tidyr"           "/usr/local/lib/R/site-library"

tidyselect      "tidyselect"      "/usr/local/lib/R/site-library"

tidyverse       "tidyverse"       "/usr/local/lib/R/site-library"

tinytex         "tinytex"         "/usr/local/lib/R/site-library"

tm              "tm"              "/usr/local/lib/R/site-library"

tmap            "tmap"            "/usr/local/lib/R/site-library"

tmaptools       "tmaptools"       "/usr/local/lib/R/site-library"

txtplot         "txtplot"         "/usr/local/lib/R/site-library"

units           "units"           "/usr/local/lib/R/site-library"

usethis         "usethis"         "/usr/local/lib/R/site-library"

utf8            "utf8"            "/usr/local/lib/R/site-library"

uuid            "uuid"            "/usr/local/lib/R/site-library"

V8              "V8"              "/usr/local/lib/R/site-library"

viridis         "viridis"         "/usr/local/lib/R/site-library"

viridisLite     "viridisLite"     "/usr/local/lib/R/site-library"

webshot         "webshot"         "/usr/local/lib/R/site-library"

whisker         "whisker"         "/usr/local/lib/R/site-library"

withr           "withr"           "/usr/local/lib/R/site-library"

wordcloud2      "wordcloud2"      "/usr/local/lib/R/site-library"

xfun            "xfun"            "/usr/local/lib/R/site-library"

XML             "XML"             "/usr/local/lib/R/site-library"

xml2            "xml2"            "/usr/local/lib/R/site-library"

xopen           "xopen"           "/usr/local/lib/R/site-library"

xtable          "xtable"          "/usr/local/lib/R/site-library"

xts             "xts"             "/usr/local/lib/R/site-library"

yaml            "yaml"            "/usr/local/lib/R/site-library"

zoo             "zoo"             "/usr/local/lib/R/site-library"

base            "base"            "/usr/lib/R/library"           

boot            "boot"            "/usr/lib/R/library"           

class           "class"           "/usr/lib/R/library"           

cluster         "cluster"         "/usr/lib/R/library"           

codetools       "codetools"       "/usr/lib/R/library"           

compiler        "compiler"        "/usr/lib/R/library"           

datasets        "datasets"        "/usr/lib/R/library"           

foreign         "foreign"         "/usr/lib/R/library"           

graphics        "graphics"        "/usr/lib/R/library"           

grDevices       "grDevices"       "/usr/lib/R/library"           

grid            "grid"            "/usr/lib/R/library"           

KernSmooth      "KernSmooth"      "/usr/lib/R/library"           

lattice         "lattice"         "/usr/lib/R/library"           

MASS            "MASS"            "/usr/lib/R/library"           

Matrix          "Matrix"          "/usr/lib/R/library"           

methods         "methods"         "/usr/lib/R/library"           

mgcv            "mgcv"            "/usr/lib/R/library"           

nlme            "nlme"            "/usr/lib/R/library"           

nnet            "nnet"            "/usr/lib/R/library"           

parallel        "parallel"        "/usr/lib/R/library"           

rpart           "rpart"           "/usr/lib/R/library"           

spatial         "spatial"         "/usr/lib/R/library"           

splines         "splines"         "/usr/lib/R/library"           

stats           "stats"           "/usr/lib/R/library"           

stats4          "stats4"          "/usr/lib/R/library"           

survival        "survival"        "/usr/lib/R/library"           

tcltk           "tcltk"           "/usr/lib/R/library"           

tools           "tools"           "/usr/lib/R/library"           

utils           "utils"           "/usr/lib/R/library"    

       

error logs can be found in /var/log/shiny-server/



last revision  3/4/19