Archives for posts with tag: NICAR

One of the most popular posts on Ricochet was the collection of dataviz tools, slides and links from last year’s NICAR conference.

It was so popular, in fact, that people have asked me to make a similar collection again. So from Feb. 23–26, I’ll be updating this post with all the great things NICARians have to share this year.

Follow #NICAR12 on Twitter for the buzz; come to this page for the goods. And if you’re attending the conference, be sure to buy a T-shirt to support IRE, the organization that puts this fantastic event together. Ben Welsh of The Los Angeles Times is taking candid photos and posting them on Flickr.

Have links from sessions you attended? Post them in comments or ping me on Twitter @MacDiva and I’ll add them to this list.

Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

Presentations & Tutorials


Bringing Maps to Fruition (from Michelle Minkoff)
Free tools for scraping data without programming (from Chris Keller and Michelle Minkoff)
Instructions for Hands-on Web Scraping Without Programming (from Chris Keller and Michelle Minkoff)
Locating the Story: The Latest in Online Maps and mapping links (from Ben Welsh)
Mapping links & presentation (from David Herzog)
Social Media Sleuthing (from Doug Haddix)
freeDive Tips & Tricks (from the Knight Digital Media Center)
CAR on a Shoestring (from Kevin Crowe, Patrick Sweet and Mary Jo Webster)
Regular Expressions: An Introduction (from Kevin Crowe, Patrick Sweet and Mary Jo Webster)
Create a moderation form using Google Forms and Fusion Tables
Scraping with Django (from Kevin Schaul)
How to turn PDFs into a searchable, sortable table (from Kevin Schaul)
Get the Most Out of Fusion Tables (from Rebecca Shapley)
Data viz in 20 minutes: jQuery DataTables (from Christopher Schnaars)
How to set up Python in Windows 7 (from Anthony DeBarros)
Data visualization best practices (from Kat Downs)
NodeXL for Network analysis (from Peter Aldhous)
Network Analysis for News (from Peter Aldhous and Peggy Heinkel-Wolfe)
Network analysis for news (video of Peter Aldhous’s NICAR12 talk)
How to Use Google Refine for Investigative Journalism (from Dan Nguyen)
Mapping is for Everyone – How to make all kinds of maps (from Sharon Machlis)
Advanced Excel techniques tipsheet (from MaryJo Webster)
How do you edit a story made of software? (from Alexander Howard)
Election Night Results & Maps (from John Keefe)
Covering Elections presentation (from Al Shaw)
Making friends with map projections (from Ben Welsh and Michael Corey)
Database validation (from JT Johnson)
Web scraping with Node.js (from Al Shaw)
Who is John Doe — and where to get the paper on him
Practical TastyPie for the Modern Djangonaut (from Jeremy Bowers)
Weathering the Storm: Using data to bolster the traditional weather story (from Stephen Stirling)
Build your first Django news app (from the IRE NICAR12 Django workshop)
GeoCommons walkthrough (from Paul Monies)
QGIS 1 workshop tutorial (from Michael Corey)
Tell Me a Story! – storytelling and data journalism (from Anthony DeBarros)
Human-assisted reporting: How to create robot reporters in your own image (from Ben Welsh)
How I learned to stop worrying and love flat files (from Ben Welsh)
Infect the CMS (from Jacob Harris)
Inspect the Web With Your Browser’s Web Inspector (from Dan Nguyen)
An Intro to R (from Jacob Fenton)
Slides from “Mapping is Hard” (from Brian Boyer)
TileMill hands-on tutorial (from Chris Amico, Brian Boyer and Matt Stiles)
Own Your Map Stack (from Chris Amico, Brian Boyer and Matt Stiles)
Natural Language Toolkit (NLTK) basics (from Jacob Perkins)
Connecting to state data using OpenMissouri.org (from David Herzog)
How to convert PDFs to Excel in Windows (from IRE)
Quantum GIS (QGIS) 2 workshop (from Michael Corey)
How to turn PDFs into text (from Dan Nguyen)
Web scraping in Python workshop tutorial (from Mark Ng)
Infiltrate the Ad Department (from Ryan Pitts)
Map Graphics for Video (from Michael Corey)
What We Can Find Out from Elections (from Aaron Bycoffe)
The Latest in Mapping with Javascript and jQuery (from Timothy Barmann)
How to Make a PANDA (from Brian Boyer)
The Farenthold Surprise (election panel presentation from Derek Willis)
Displaying data geographically: Creating a one-layer map in ArcMap (from Tom Meagher)
An intro to csvKit (from Christopher Groskopf and Anthony DeBarros)
Integrating CAR into a daily Beat (from Kate Martin)
How to use the SIMILE Exhibit timeline framework (from David Karger)
Tableau training handouts (from Tableau)
CAR Training 2012 including mapping data sets, practice data sets and tip sheets (from Jennifer LaFleur)


Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

Software & Tools


Twazzup – find breaking news, popular hashtags, influential users
Reporters’ Lab Reviews – a link list of tools, techniques and research for public affairs reporting
Twellow – a yellow pages for Twitter
Twiangulate – find sources and groups of people on Twitter
Crowdbooster – monitor and analyze buzz on social media sites
KnowEm Username Search – finds the social networks a person or organization/brand is using
Muckrack Pro – add yourself to the list of journalists or find journalists covering a particular topic
The Archivist – save tweets and export to Excel to analyze later
PowerPivot for Excel – “Load massive amounts of data from virtually any source, process in seconds and model with powerful analytical capabilities”
Pandoc – a universal document converter
HTML-to-PDF – converts HTML to PDF docs for free
Mr. Data Converter – converts Excel data into one of several Web-friendly formats, including HTML, JSON and XML.
Natural Language Toolkit – for machine language text analysis
Voyant Tools – Web-based document analysis
ClearForest Gnosis – Firefox plugin that uses OpenCalais for data extraction
Exhibit – a publishing framework for data-rich interactive web pages
DocumentCloud – store, analyze and annotate PDFs
DataTables – jQuery plugin to create sortable datasets
Ben Welsh’s triumvirate of tools that allow you to copy Google Maps’ functionality:
   – a data source, like OpenStreetMap
   – a tile set, like what you can make with TileMill
   – a JavaScript interface, like Leaflet
OpenOffice – open source office suite software (word processor, spreadsheet, presentation/slide deck, database)
QGIS – Open source geographic information system
Shape to Fusion (a.k.a. Shpescape) – Import shapefiles to Fusion Tables
MySQL – Database software
Google Refine – data cleaner
Junar – Discover and track data
The Overview Project
Visicheck – ensures your graphics are visible to the colorblind
Colorbrewer – in case you need help with color schemes for your design
Color Oracle – colorblindness simulator for Mac OS, Windows and Linux
0 to 255 – find variations of any color
Beautiful Soup – useful for many things, including parsing HTML
Weave – Web-based analysis and visualization environment. Made by a partnership between the University of Massachusetts Lowell and Open Indicators Consortium
Highcharts – create interactive JavaScript charts (free for non-commercial use)
Indiemapper – Upload shapefiles and convert them to create static, thematic maps
CSV-to-JSON converter
Sinatra a lightweight Ruby/Rails framework for creating apps
• Use Google Docs, XPath and the =importxml() function to put data in a spreadsheet
PANDA Project
Timemap syncs a SIMILE timeline to a web-based map
Tabletop – allows you to use Google spreadsheets as your app backend
Js2Coffee – converts Javascript to CoffeeScript and back
CoffeeScript sandbox
iPL2 – ask a librarian, search through the Internet Public Library (IPL) and the Librarians’ Internet Index (LII) websites.
• “Lesson of the night: Want to put census geos in fusion tables? Keep it stupid simple: convert US Census data from TIGER into shape files with shpescape” — tip from Matt Kiefer
Rubular – a Ruby regular expression editor
Timeline Setter – makes timelines from spreadsheets
Spoofcard changes your voice and gives you a temporary phone number
Tablechart turns HTML tables into charts
Spam Mimic – hide a message in spam
FEC scraper/FEC parser – Chris Schnaars’ script on Github

Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

References


• The American Library Association’s wiki of government databases (from Dan Nguyen)
Penn Treebank Project reference – Use it in conjunction with the Natural Language Toolkit (NLTK)
Geomedia Google Group
NICAR-L mailing list
Google Public Data Explorer
InfoVis Wiki – a catchcall list of papers, conferences, patterns and jobs in information visualization
Spatial Reference – an IMDB-like catalog of spatial reference systems
22 free visualization tools collected by ComputerWorld
Free Data Visualization tools – a collection from Sharon Machlis
8 cool tools for data analysis, visualization and presentation (from Sharon Machlis)
Chart and image gallery: 30 free tools for data visualization and analysis (from Sharon Machlis)
LocalHealthData.org – find health data from more than 70 sources and 300+ datasets
Analytic Journalism “It’s not ‘all about story’ if you don’t have anything to say.”
How to install MySQL and Navicat on Windows
Freebase – an entity graph/Wikipedia-like collection of data
Save the Post Office – records U.S. post office consolidations and closures
• Los Angeles Times datadesk Github repository with code for you to use
USASpending.gov – Official record of Federal Funding Accountability and Transparency Act (Transparency Act)
&bull: Data for the Public Good by Alexander Howard (free eBook)
CongressionalPrimaries.org shows what Illinois congressional candidates are tweeting about
Civic Commons Marketplace collects open government efforts in the U.S.
OpenCorporates is in the process of collecting information on every corporate entity in the world
• USA Today’s Developer Network

Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

Work Samples


Bailed out banks profit from tax liens (Arizona Star heat maps showed property locations, making the story very clear)
Race gap found in traffic stops (Milwaukee Journal-Sentinel showed the racial disparity in pullovers and on further examination, municipal maintenance requests)
Texas redistricting map and slider code (Texas Tribune)
The Poverty Gap shows a clear correlation between poverty and access to education (ProPublica)
2012 Election Results big board, one approach to visual presentation of election info that tells you the story of the election immediately (The New York Times)
Little Loving County grabs a bit of Texas’ growth a census story unlike the usual census stories (The Dallas Morning News)
Riot rumours: how misinformation spread on Twitter during a time of crisis uses data analysis to watch the spread and suppression of rumors about the London riots (The Guardian)
Discover Boston Public Schools (Code for America)
SchoolBook makes teacher data reports for New York City schools
Redistricting: New lines leave some voters without a senator (The [Riverside, Calif.] Press-Enterprise)

Jump to Tutorials | Software & Tools | References | Work Samples

And finally, no journalism nerdfest would be complete without a demonstration of the latest hotness: Drone journalism by Matt Waite.

Drone Journalism Demo – Matt Waite from John Keefe on Vimeo.

CAR 2011 was stuffed full of information, so much so that the only way to keep up with everything has been to keep a log of what people have been sharing.

Feb. 28 update: Thanks to everyone who’s forwarded additional links and presentations (I’m marking them with NEW as they’re added) and to all who’ve sent me nice notes about this list.

Philip Smith forwarded a JSON file of NICAR tweets with links in them. Want it? Download it.

This year’s conference looks to have been a tremendous success, bringing in the most registered attendees in nearly a decade. Congratulations to NICAR for a terrific, educational and inspiring event.

A more narrative look at what happened at the conference can be found on the conference blog. But if you’re anxious to dive in, this is your buffet: Prepare to have your mind blown.

Got links from sessions you attended? Post them in comments or ping me on Twitter @MacDiva and I’ll add them to this list.

Jump to Tutorials | Software & Tools | References | Work Samples
 

Presentations & Tutorials

NEW Using TwitInfo and TweeQL to find and tell stories (from Adam Marcus)

NEW A Gentle Introduction to SQL using SQLite: slides, full tutorial and steps only (from Troy Thibodeaux)

Valet Parking Your Django App (from Jeremy Bowers)

Similarity algorithms using Python (from Luke Rosiak)

The Quick and Dirty Varnish Setup for Django (from Andy Boyle)

Making HTML Tables Interactive (from Michelle Minkoff)

View more presentations from Michelle Minkoff

QuantumGIS 1 tutorial and files (16.3MB) (from Timothy Barmann)
• Use JavaScript and jQuery to create interactive maps: tutorial and files (17.5 MB) (from Timothy Barmann)
• How to break news online – and use LA Times app engine tools (from Ben Welsh)

NodeXL for Social Network Analysis (from Peter Aldhous)

• Excel, CAR and mapping training tipsheets, slides and datasets (from Jennifer LaFleur)

My Favorite (Excel) Things (from MaryJo Webster)

Latest in Mapping tools and examples

(Ruby) Coding for Absolute Beginners (from Dan Nguyen) – following the tutorial will produce “My Very First Web Page

Google Refine tutorial and datasets (that download to your hard drive on click) (from David Huynh)

APIs: Making the Web a Data Medium (from Anthony DeBarros and Derek Willis)

NEW R for Statistics: First Steps (from Peter Aldhous)

NEW R for Statistics: Automate Your Analysis (from Jacob Fenton)

Hands-on R, a step-by-step tutorial (from Jacob Fenton)

Ruby4Kids — mentioned in passing as a low-friction way to learn the basics of Ruby

How to make an intensity map with custom boundaries using Google Fusion Tables

Google Fusion Tables tutorial

Cracking Open Electric Records slides and case law < = link launches a PDF bundle (from DB Smallman) • Internet Reporting: What You Should Know (from Jack Gillum)

• Free software: From Spreadsheets to GIS, Part 1 and Part 2 (from Jacob Fenton and Anthony DeBarros)

Beyond Mapping: Spatial Analysis on the Cheap (from Long Creative)

Beautiful Data (from Aron Pilhofer)

View more presentations from pilhofer

• Intro to Python, Session 1 tutorial and Python tipsheet (from Jacqueline Kazil and Serdar Tumgoren)

Getting into a data-oriented mindset (from Mary Jo Webster and Wendell Cochran)

Dataviz for beginners (from Matt Stiles and Sanjay Bhatt)

MGRS Explained (from Jacob Harris)

Data Visualization with JavaScript and HTML5 (from Jeff Larson)

Tutorial: Census Data with Tableau Public

PostGIS is Your New Bicycle – be wowed by a free alternative to costly desktop GIS (from Mike Corey and Ben Welsh)


 

Software & Tools

Jump to Tutorials | Software & Tools | References | Work Samples

3Scale – API management and monetization tool (free trial)
API Playground – try APIs, no coding skills necessary
Backbone.js adds a models-collections-views structure to JavaScript applications
BatchGeo interactive map maker
Biznar.com – business search engine
CanIUse.com browser compatibility tables
Census Block Conversions API
ChangeTracker from ProPublica – track changes to any website
ChinaVitae – learn who’s who in power in China
CollegeInsight – compare universities by cost, financial aid, diversity, job placement rate
DataWrangler cleans and transforms data
• Download manager downTHEMall is a FireFox extension that grabs webpage links and images.
Europe Media Monitor’s NewsBrief – an international alternative to Google News
EUROCONTROL – “find blocked private planes that might have flown to Europe, for example: see which executives are going to Cannes”
FCC Census Block Conversions API – boundary service API, excellent for mapping
• The FireShot FireFox extension creates browser screenshots, adds annotation and more.
Foreign Labor Certification Data Center – find what visas a company has applied for (there may be wage information tied to the application)
Get Lat Lon – finds latitude and longitude for any location worldwide
• Free Google Drawings wireframe templates
Google Fusion Tables for data analysis and visualization
Google Refine for data cleaning
Inmarsat Ships Directory – lookup a ship’s phone number
JSFiddle online JavaScript editor
Jigsaw: “Visual analytics for exploring and understanding document collections”
Little Sis – visualizing the networks of social, financial and political power
MarineTraffic.com – track vessels in real time
Mayan open source, Django-based document manager
Mr. Data Converter converts Excel data into web-friendly formats
Needlebase
NETROnline – public records search, especially good for real property lookups
NodeXL uses Excel for network analysis
NodeXL Teaching lessons and tutorials
Numberway.com – lookup phone numbers around the world
Outwit Hub – FireFox plugin for scraping websites
PDFonFly – converts web pages to PDFs
PhraseNet diagrams relationships between words in text
PostGIS – adds mapping ability to PostgreSQL
PrivacyChoice – rates website privacy policies
Protovis
PySAL an open source Python library for spatial analysis functions
R statistical analysis software
R libraries recommended by Amanda Cox, Jeff Larson and others: ggplot, RColorBrewer (color picker), rgdal (bindings for GDAL – the Geospatial Data Abstraction Library), survival (survival analysis)
Recorded Future – temporal analysis search engine uses predictive analytics to discover the likelihood of events in the future
RSRuby – use the R environment in your Ruby program
Rubular – test your regex on the fly
Simile Timeline
Scraper Wiki
Snitch.Name – people lookup
Tableau Public
TimeFlow
TinEye finds information on uploaded images, including usage, higher resolutions, modified versions
Tweeql access the Twitter API by using SQL syntax (requires Python)
TwitInfo chart Twitter keyword frequency and sentiment
USA Spending – see what the US government is spending money on
 

References

Jump to Tutorials | Software & Tools | References | Work Samples

NEW Journalists learning Python Google group
NICAR ‘Net Tour – an index of links from IRE for watchdog research and learning computer-assisted reporting
The New Precision Journalism (from Philip Meyer)
The Logic Of Causal Order by James A. Davis (recommended by Philip Meyer)
• US Government Health Data
Health Indicators Warehouse
Coordinate Systems Overview for mapping
Concepts of Probability (statistics!)
Advanced Probability and Statistics, 2nd Ed. by the CK-12 Foundation
Thomas Lumley: work page (statistics! and Amanda Cox’s professor)
Hadley Wickham (statistics! and the maker of ggplot for R)
Graphical Inference for Infovis by Hadley Wickham, Dianne Cook, Heike Hofmann and Andreas Buja (“How do we know if what we see (in a data visualization) is really there?”)
“Be Careful What You Do With That Cell Phone Recording; It Could Land You in Jail” (from DB Smallman)
Gary’s Social Media Count – see the volume of social media activity
Quantitative Discovery from Qualitative Information: A General-Purpose Document Clustering Methodology by Gary King
Producing Online News: Digital Skills, Stronger Stories by Ryan Thornburg
US State Department Foreign Affairs Manual, section on information security, a.k.a. 12 FAM 500
Five Databases in 50 Minutes: Government Session (from the CAR2011 conference blog)
News Apps: What Works and Why (from the CAR2011 conference blog)
Analysis-ready census data (from USA Today, available to NICAR members only)
A directory of statistics bureaus by country (from Statistics Sweden)
Data Visualization for Beginners (from the CAR2011 conference blog)
Tracking the Economy and Business (from the CAR2011 conference blog)
Benford’s Law (statistics!)

 

Work Samples

Jump to Tutorials | Software & Tools | References | Work Samples

• The Wall Street Journal investigative report, “Confidentiality Cloaks Medicare Abuse” with database created by Mo Tamman
• The Center for Public Integrity investigative report, “Unproven for Older Women, Digital Mammography Saps Medicare Dollars
The Year in CAR
• Des Moines Register potholes map
FlyOnTime.us – find the most on-time flights between cities (uses US government raw data)
Employment Market Explorer – find out what the local employment market looks like. Compare local, regional and national rates and labor market dynamics. (uses US government raw data)
WildTrack – using data to monitor endangered species populations
• Roundup of state-based 2010 census stories
The Killing Roads – interactive map of highway accidents in Norway
• The entire King James Bible as a word tree
Who Runs HK – network graph of the people in power in Hong Kong
Research by Martin Wattenberg, including the highlighted works, Name Voyager, Map of the Markets, Shape of Song and Fleshmap

Jump to Tutorials | Software & Tools | References | Work Samples