Archives for category: General Journalism

Ben Welsh of the Los Angeles Times Data Desk spoke at the International Symposium on Online Journalism in Austin yesterday, around the same time that I was speaking on a panel about data journalism with Erik Hinton (@erikhinton), Al Shaw (@A_L) and Andrei Scheinkman (@acheink) at NYU Local Young Media Weekend.

Ben gave this talk at NICAR in St. Louis earlier this year. Lucky for us, ISOJ streamed it, and La Nacion’s data team captured it.

Watch, learn, and dig deeper in Ben’s Delicious stack. Ben also writes terrific material on his site, Palewire, and tweets at @palewire.

One of the most popular posts on Ricochet was the collection of dataviz tools, slides and links from last year’s NICAR conference.

It was so popular, in fact, that people have asked me to make a similar collection again. So from Feb. 23–26, I’ll be updating this post with all the great things NICARians have to share this year.

Follow #NICAR12 on Twitter for the buzz; come to this page for the goods. And if you’re attending the conference, be sure to buy a T-shirt to support IRE, the organization that puts this fantastic event together. Ben Welsh of The Los Angeles Times is taking candid photos and posting them on Flickr.

Have links from sessions you attended? Post them in comments or ping me on Twitter @MacDiva and I’ll add them to this list.

Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

Presentations & Tutorials


Bringing Maps to Fruition (from Michelle Minkoff)
Free tools for scraping data without programming (from Chris Keller and Michelle Minkoff)
Instructions for Hands-on Web Scraping Without Programming (from Chris Keller and Michelle Minkoff)
Locating the Story: The Latest in Online Maps and mapping links (from Ben Welsh)
Mapping links & presentation (from David Herzog)
Social Media Sleuthing (from Doug Haddix)
freeDive Tips & Tricks (from the Knight Digital Media Center)
CAR on a Shoestring (from Kevin Crowe, Patrick Sweet and Mary Jo Webster)
Regular Expressions: An Introduction (from Kevin Crowe, Patrick Sweet and Mary Jo Webster)
Create a moderation form using Google Forms and Fusion Tables
Scraping with Django (from Kevin Schaul)
How to turn PDFs into a searchable, sortable table (from Kevin Schaul)
Get the Most Out of Fusion Tables (from Rebecca Shapley)
Data viz in 20 minutes: jQuery DataTables (from Christopher Schnaars)
How to set up Python in Windows 7 (from Anthony DeBarros)
Data visualization best practices (from Kat Downs)
NodeXL for Network analysis (from Peter Aldhous)
Network Analysis for News (from Peter Aldhous and Peggy Heinkel-Wolfe)
Network analysis for news (video of Peter Aldhous’s NICAR12 talk)
How to Use Google Refine for Investigative Journalism (from Dan Nguyen)
Mapping is for Everyone – How to make all kinds of maps (from Sharon Machlis)
Advanced Excel techniques tipsheet (from MaryJo Webster)
How do you edit a story made of software? (from Alexander Howard)
Election Night Results & Maps (from John Keefe)
Covering Elections presentation (from Al Shaw)
Making friends with map projections (from Ben Welsh and Michael Corey)
Database validation (from JT Johnson)
Web scraping with Node.js (from Al Shaw)
Who is John Doe — and where to get the paper on him
Practical TastyPie for the Modern Djangonaut (from Jeremy Bowers)
Weathering the Storm: Using data to bolster the traditional weather story (from Stephen Stirling)
Build your first Django news app (from the IRE NICAR12 Django workshop)
GeoCommons walkthrough (from Paul Monies)
QGIS 1 workshop tutorial (from Michael Corey)
Tell Me a Story! – storytelling and data journalism (from Anthony DeBarros)
Human-assisted reporting: How to create robot reporters in your own image (from Ben Welsh)
How I learned to stop worrying and love flat files (from Ben Welsh)
Infect the CMS (from Jacob Harris)
Inspect the Web With Your Browser’s Web Inspector (from Dan Nguyen)
An Intro to R (from Jacob Fenton)
Slides from “Mapping is Hard” (from Brian Boyer)
TileMill hands-on tutorial (from Chris Amico, Brian Boyer and Matt Stiles)
Own Your Map Stack (from Chris Amico, Brian Boyer and Matt Stiles)
Natural Language Toolkit (NLTK) basics (from Jacob Perkins)
Connecting to state data using OpenMissouri.org (from David Herzog)
How to convert PDFs to Excel in Windows (from IRE)
Quantum GIS (QGIS) 2 workshop (from Michael Corey)
How to turn PDFs into text (from Dan Nguyen)
Web scraping in Python workshop tutorial (from Mark Ng)
Infiltrate the Ad Department (from Ryan Pitts)
Map Graphics for Video (from Michael Corey)
What We Can Find Out from Elections (from Aaron Bycoffe)
The Latest in Mapping with Javascript and jQuery (from Timothy Barmann)
How to Make a PANDA (from Brian Boyer)
The Farenthold Surprise (election panel presentation from Derek Willis)
Displaying data geographically: Creating a one-layer map in ArcMap (from Tom Meagher)
An intro to csvKit (from Christopher Groskopf and Anthony DeBarros)
Integrating CAR into a daily Beat (from Kate Martin)
How to use the SIMILE Exhibit timeline framework (from David Karger)
Tableau training handouts (from Tableau)
CAR Training 2012 including mapping data sets, practice data sets and tip sheets (from Jennifer LaFleur)


Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

Software & Tools


Twazzup – find breaking news, popular hashtags, influential users
Reporters’ Lab Reviews – a link list of tools, techniques and research for public affairs reporting
Twellow – a yellow pages for Twitter
Twiangulate – find sources and groups of people on Twitter
Crowdbooster – monitor and analyze buzz on social media sites
KnowEm Username Search – finds the social networks a person or organization/brand is using
Muckrack Pro – add yourself to the list of journalists or find journalists covering a particular topic
The Archivist – save tweets and export to Excel to analyze later
PowerPivot for Excel – “Load massive amounts of data from virtually any source, process in seconds and model with powerful analytical capabilities”
Pandoc – a universal document converter
HTML-to-PDF – converts HTML to PDF docs for free
Mr. Data Converter – converts Excel data into one of several Web-friendly formats, including HTML, JSON and XML.
Natural Language Toolkit – for machine language text analysis
Voyant Tools – Web-based document analysis
ClearForest Gnosis – Firefox plugin that uses OpenCalais for data extraction
Exhibit – a publishing framework for data-rich interactive web pages
DocumentCloud – store, analyze and annotate PDFs
DataTables – jQuery plugin to create sortable datasets
Ben Welsh’s triumvirate of tools that allow you to copy Google Maps’ functionality:
   – a data source, like OpenStreetMap
   – a tile set, like what you can make with TileMill
   – a JavaScript interface, like Leaflet
OpenOffice – open source office suite software (word processor, spreadsheet, presentation/slide deck, database)
QGIS – Open source geographic information system
Shape to Fusion (a.k.a. Shpescape) – Import shapefiles to Fusion Tables
MySQL – Database software
Google Refine – data cleaner
Junar – Discover and track data
The Overview Project
Visicheck – ensures your graphics are visible to the colorblind
Colorbrewer – in case you need help with color schemes for your design
Color Oracle – colorblindness simulator for Mac OS, Windows and Linux
0 to 255 – find variations of any color
Beautiful Soup – useful for many things, including parsing HTML
Weave – Web-based analysis and visualization environment. Made by a partnership between the University of Massachusetts Lowell and Open Indicators Consortium
Highcharts – create interactive JavaScript charts (free for non-commercial use)
Indiemapper – Upload shapefiles and convert them to create static, thematic maps
CSV-to-JSON converter
Sinatra a lightweight Ruby/Rails framework for creating apps
• Use Google Docs, XPath and the =importxml() function to put data in a spreadsheet
PANDA Project
Timemap syncs a SIMILE timeline to a web-based map
Tabletop – allows you to use Google spreadsheets as your app backend
Js2Coffee – converts Javascript to CoffeeScript and back
CoffeeScript sandbox
iPL2 – ask a librarian, search through the Internet Public Library (IPL) and the Librarians’ Internet Index (LII) websites.
• “Lesson of the night: Want to put census geos in fusion tables? Keep it stupid simple: convert US Census data from TIGER into shape files with shpescape” — tip from Matt Kiefer
Rubular – a Ruby regular expression editor
Timeline Setter – makes timelines from spreadsheets
Spoofcard changes your voice and gives you a temporary phone number
Tablechart turns HTML tables into charts
Spam Mimic – hide a message in spam
FEC scraper/FEC parser – Chris Schnaars’ script on Github

Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

References


• The American Library Association’s wiki of government databases (from Dan Nguyen)
Penn Treebank Project reference – Use it in conjunction with the Natural Language Toolkit (NLTK)
Geomedia Google Group
NICAR-L mailing list
Google Public Data Explorer
InfoVis Wiki – a catchcall list of papers, conferences, patterns and jobs in information visualization
Spatial Reference – an IMDB-like catalog of spatial reference systems
22 free visualization tools collected by ComputerWorld
Free Data Visualization tools – a collection from Sharon Machlis
8 cool tools for data analysis, visualization and presentation (from Sharon Machlis)
Chart and image gallery: 30 free tools for data visualization and analysis (from Sharon Machlis)
LocalHealthData.org – find health data from more than 70 sources and 300+ datasets
Analytic Journalism “It’s not ‘all about story’ if you don’t have anything to say.”
How to install MySQL and Navicat on Windows
Freebase – an entity graph/Wikipedia-like collection of data
Save the Post Office – records U.S. post office consolidations and closures
• Los Angeles Times datadesk Github repository with code for you to use
USASpending.gov – Official record of Federal Funding Accountability and Transparency Act (Transparency Act)
&bull: Data for the Public Good by Alexander Howard (free eBook)
CongressionalPrimaries.org shows what Illinois congressional candidates are tweeting about
Civic Commons Marketplace collects open government efforts in the U.S.
OpenCorporates is in the process of collecting information on every corporate entity in the world
• USA Today’s Developer Network

Jump to Presentations & Tutorials | Software & Tools | References | Work Samples
 

Work Samples


Bailed out banks profit from tax liens (Arizona Star heat maps showed property locations, making the story very clear)
Race gap found in traffic stops (Milwaukee Journal-Sentinel showed the racial disparity in pullovers and on further examination, municipal maintenance requests)
Texas redistricting map and slider code (Texas Tribune)
The Poverty Gap shows a clear correlation between poverty and access to education (ProPublica)
2012 Election Results big board, one approach to visual presentation of election info that tells you the story of the election immediately (The New York Times)
Little Loving County grabs a bit of Texas’ growth a census story unlike the usual census stories (The Dallas Morning News)
Riot rumours: how misinformation spread on Twitter during a time of crisis uses data analysis to watch the spread and suppression of rumors about the London riots (The Guardian)
Discover Boston Public Schools (Code for America)
SchoolBook makes teacher data reports for New York City schools
Redistricting: New lines leave some voters without a senator (The [Riverside, Calif.] Press-Enterprise)

Jump to Tutorials | Software & Tools | References | Work Samples

And finally, no journalism nerdfest would be complete without a demonstration of the latest hotness: Drone journalism by Matt Waite.

Drone Journalism Demo – Matt Waite from John Keefe on Vimeo.

If you want to understand someone, my advice is to sit next to them and solve a very hard problem together. You will learn who they are by watching how they think.
— Michael Lopp

PyGotham and the Q&A that followed, I’m finding more reasons than ever to read Michael Lopp’s books and blog, Rands in Repose.

The tension between those who make digital products and those who don’t is a systemic problem that seems to stymie every industry, yet so few people know how to resolve it — and resolve it at scale. There must be a collection of good advice somewhere. If not, it’s probably time to start one. What do you say?

(Photo: Ed Yourdon/Flickr)

Last week, Alastair Dant, lead interactive technologist at The Guardian, came to Hacks/Hackers NYC to show how his team produces its informative and award-winning interactive graphics.

It’s a wide-ranging talk about what’s new and inspiring about news technology, and how each team member’s unique skills contribute to the whole.

Well worth watching. And if you want to deeply nerd out with The Guardian, check out their Developer Blog.

The projects mentioned in Alastair’s talk:

Alastair’s team is Martin Shuttleworth, Mariana Santos, Jonathan Richards and Alex Graul.

Quora product designer Rebekah Cox answers the question (on Quora, naturally).

Note, this is one person’s perspective. Nevertheless, her response is a really important read. For some time now, I’ve been studying how to increase the number of women in tech and as I see it, the hurdles are three: Culture. Education. Mentorship/Role Models.

Of the three, example after example shows culture is the hardest to overcome. Cox illuminates what tech culture is like — and offers valuable advice on how a woman can use perspective to her advantage.

It’s my hope that tech culture will evolve in a way that doesn’t require a hard shell to stay in and excel.

But until then, people who don’t feel like alpha nerds but want to be in tech can learn a lot from Rebekah Cox’s reply and footnotes. Read it and leap.

The future belongs to the makers. I don’t know who said it first, but the more time I spend looking at the world out there, the more I believe this to be true. We can dream great dreams, but only those concepts made concrete can be tried, tested and built upon.

I think that’s why people like hackathons. This year, I’m cohosting Hacks/Hackers Hacking @ ONA11, a hack day in Boston on Sept. 22, the day before the start of the Online News Association conference. You’re invited to join 100 like-minded makers of all stripes by signing up now. It’s $20 — a small price to pay for what you’ll get out of it.

September is a long way off, which means there’s lots of time for pre-hackathon planning and collaboration. Whether or not you’ve participated in a hackathon before (and if you haven’t, here’s why you should), you can help make the period from the signup announcement up to day of event a productive and collaborative one.

Post your suggestions for bringing any hackathon community together online at the Hackathon Runway EtherPad instance, or feel free to leave your thoughts in comments below.

Things I’ve been thinking about:

  • What’s the most effective way to collaborate?
  • How can non-coders play an active role?
  • What tools (software, SDKs, repos, APIs, apps) have you used?
  • How can the hacks made be incorporated into everyday use?

Your ideas and feedback are always appreciated.

By the way, I’m here at the MIT Knight Civic Media Conference, one of the most exciting annual gatherings for people interested in gathering, organizing and disseminating public information. It’s the highlight of the Knight News Challenge, and showcases some of the most motivated and dedicated thinkers and doers in the field. Sixteen proposals were funded this year, and they’re the most wide-ranging and potentially impactful yet.

If you’re here, say hi and be sure to come to my 2:30 p.m. unconference session today on building pre-hackathon community. I’ll be there with Matt Carroll of the Boston Globe and Phillip Smith of the Mozilla Foundation.

Speaking of community, I’ll be on Poynter.org at 3 p.m. ET to answer your questions about the skills journalists need to effectively engage audiences. Join me, won’t you?

Thanks to Joe Grimm and Mallary Tenore for inviting me. You can read the replay in the window below.

If you didn’t get to Metafilter founder Matt Haughey’s talk about community management at SXSW, don’t panic. Once he got home, he recorded it with slides in front of his computer and posted it online.

Thanks, Matt!

SXSW this year was huge — so big it was inevitable that sessions worth seeing were missed. Fortunately, there’s video.

Here, designer Khoi Vinh talks with design writer Alissa Walker about where design for online reading is headed.

If you don’t have time to watch the video, read Alissa’s write-up on Fast Company.

A long time ago in an Internet culture far, far from where we are now, it was normal to use a handle online. As the founder of a Macintosh enthusiast webzine (remember those?), I was known as MacDiva. People liked the name: It was catchy, meaningful and easy to remember.

In 2007, like many of you, I discovered Twitter. That same year, I joined the Online News Association conference planning committee with the intention of sharing what was happening with as many people as possible, even if they couldn’t be there in person.

At the time, I couldn’t post to the conference website; that was reserved as the showcase for the student newsroom. I couldn’t post to the ONA website; that would have presented problems of its own. So I turned to Twitter, created @MacDivaONA and began recording what I saw.

I’ve been involved with the ONA conference planning board ever since, experimenting with different ways of bringing a virtual version of the annual event to anyone who wants to be a part of it.

The conference isn’t the only time I use Twitter, though, and ONA isn’t the only organization I’m actively involved with. And as Jennifer 8. Lee recently pointed out, people who don’t know my IRL name often do know me as MacDiva — though they don’t always remember the ONA ending.

So now that there’s a brief lull between journalism and technical events, I’m simplifying — for all of our sakes: On Twitter, I’m now @MacDiva.

Have you recently decided to change your online name? What urged you to action? Share right here, or ping me @MacDiva.