July 24, 2015 / by Open Data Portal Team / In Open Data , Data Portal

New columns in Chicago Early Learning Programs

We have added two columns to the Chicago Early Learning Programs dataset. In addition to the previously existing “Ages Served” column, there are now checkbox (true/false) columns indicating whether ages 0-3 and/or 3-5 are served.

July 16, 2015 / by Open Data Portal Team / In Open Data , Data Portal

What is a “deprecated” dataset?

The most basic answer is that it is not the dataset you want if you are looking for current data.

When possible, we try to keep historical records in a dataset with dates or other indicators that allow for filtering by time. However, in some cases – notably maps – the dataset is inherently a snapshot in time and mixing historical and new data is impossible or potentially confusing. For minor changes in the data, we usually simply overwrite the old data but for less-frequent / more-substantial changes, we often first create a copy of the old data and mark it “deprecated” in at least two ways – in the dataset title and by applying the “deprecated” topic/tag.

The deprecated dataset will be a newly created dataset so that the existing URL will continue to point to the current data.  As a result, the Created and Updated dates of the dataset can be misleading. The best indicators of the time period covered by a deprecated dataset will be the date we put in the title and the Time Period shown under “About this Dataset.”  Note that if there was a gap between when the data became outdated in a real-world sense and when we deprecated the dataset, the date in the title may refer to either.  Our deprecation process has evolved as we have gained experience and the meaning of the date should become clearer.

Separately, there is a feature built into the Data Portal that can also be used, in some cases, to see historical versions of data. Every dataset has a “More Views” button, normally used to see charts, maps, filtered views, and other presentations of the data.  However, for tabular datasets, there is also a section labeled “Dataset Snapshots.”  It often shows previous versions of the dataset.  It is important to note that the date on a snapshot is when that version of the data was replaced, not when it was created.  As a general rule, the date indicates when the next-newer snapshot (or the current version, in the case of the newest snapshot) was created. Because of technical changes in how datasets are updated, many changes no longer create snapshots and we recommend against relying on them but they can be useful where they do exist.

June 04, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Beach Water Sensors dataset cleanup

With the start of summer, we have performed some clean-up to the Beach Water Quality - Automated Sensors dataset:

• Removed records for “Weather Station” sites, which had no data and should not have been included in the first place.
• That removal made the Beach Water Quality - Automated Sensors - Records With Measurements filtered view unnecessary.  We have hidden it and will delete it altogether in no less than 30 days.  In the meantime, the URL and any API calls to it will still work.
• Broke each chart view under “More Views” into 2014 and 2015 versions, due to size and calculation/rendering time. In all cases, the 2014 version of the chart inherited the existing identifier (4x4) and URL.  (Example: Water Temperature).
• Deleted two records from 2013 with clearly erroneous (impossible) or missing measurements in all fields.

In addition, as described in a post on the Chicago Digital site, we have added two related datasets:

• Beach Weather Stations - Automated Sensors – the actual data from the blank sites removed above.
• The locations of both types of sensor site, with a mapped version.

May 30, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Dataset updates are working inconsistently - 5/30/2015

We are having problems with updates to a number of datasets.  You can confirm the last modification date of any dataset by clicking the About button in the upper right of the page.  The datasets that update multiple times per day do not appear to be affected so exact time of the last update is not relevant in this case.

We will update this post with any significant changes to the situation.

Update 6/2/2015, 10:10 am - The updates are generally working again. A notable exception is Food Inspections, which currently shows no inspections after Thursday, 5/28/2015. We are looking into the problem.

May 15, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Effects of new ward map on datasets

On 5/18/2015, the City of Chicago will formally adopt its new ward map. This is the map, based on the 2010 Census, that was used for the recent City Council elections, with the elected aldermen’s terms in office beginning on that date.

In addition to defining City Council districts, the City uses wards for many internal purposes and they appear in some datasets. The following is a list of those datasets and how they will be affected by the ward transition.

All Records Converted

In the following datasets, all records will show the new (2015) ward.

• 311 Service Requests datasets
  
• Business Licenses
• City-Owned Land Inventory (Already converted.)
  
• Crimes - 2001 to present
  
• Crimes - One year prior to present
  
• Micro-Market Recovery Program datasets
• Parking Permit Zones (”Ward - Low” and “Ward - High” will be converted. The “Sponsor Ward” column, which is ambiguous due to ward remaps and no longer used or maintained internally, will be removed altogether.)
  

Only New Records

In the following dataset, existing records will retain the current (2003) ward but records added from 5/18/2015 forward will show the new (2015) ward.

• Roadway Construction Moratoriums
  

Dual Datasets

As discussed in a recent post and one earlier this year, street sweeping zones are based around wards and therefore will change on May 18. There is one map and schedule for April through May 15 and another map and schedule for May 18 through November.

Converted at Next Update

The following datasets are updated infrequently and will retain the current (2003) ward until the next update, at which point the tentative plan is that all records will switch to the new (2015) ward.

• Bike Racks
  
• Building Code Scofflaw List
  
• Chicago Microlending Institute (CMI) Microloans
  
• CTA - System Information - Bus Stop Shelters in Digital Sign Project
  
• Parks - Buildings
  
• Parks - Locations
  
• Problem Landlord List
  
• Ward Offices (Should be updated soon, after new alderman have finalized their office locations.)
  
• Youth Centers (This dataset was replaced with Family and Support Services Delegate Agencies in October 2015, as discussed in REPLACING YOUTH CENTERS DATASET WITH FAMILY AND SUPPORT SERVICES DELEGATE AGENCIES

Please note that these plans may evolve and we will update this post, as necessary.

For historical reference, we will export the last or close-to-last versions of the affected datasets that use the current ward map to CSV and post all of these CSV files to a single ZIP archive on the Data Portal. We will add a link here once it is available. (6/19/2015 - Due to size, it was necessary to export to two ZIP files: Crimes - 2001 to present and All other datasets.)

We will also display both ward maps on many of our Data Lens pages, which are able to assign records to both sets of wards based on a Location column, even if no Ward column is available explicitly.

Updates 6/19/2015, 2:11 pm: 

• The 311 Service Request datasets moved from “Only New Records” to “All Records Converted.”
• Added the links to the ZIP archives of historical dataset exports.

Update 8/21/2017, 5:00 pm - Added note about Youth Centers dataset.

May 14, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Street sweeping schedule - 2015 - Part 2

April 23, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Updated street sweeping schedule - April/May 2015

We have updated the Street Sweeping Schedule - 2015 - April-May dataset.  There are a number of changes in the 11th Ward.

April 01, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Updating old ward map - deprecating old download files

We will be removing the standalone ward maps for 2013-2015 that exist in the Shapefile and KML format. You will be able to access and download the 2013-2015 ward map at its new location. The new location has on-screen map navigation and is also downloadable. 

The deprecated versions will be removed no earlier than April 30th.

March 27, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Street sweeping schedule - 2015

We have posted our street sweeping schedule and map, for sweeping and associated parking restrictions that begin on April 1.  These datasets go only through Friday, May 15 because Chicago’s new ward map will go into effect on May 18, affecting a range of City operations, including street sweeping.  We will post revised or supplemental datasets to cover the remainder of the street sweeping season, through November.

March 08, 2015 / by Open Data Portal Team / In Open Data , Data Portal

Modified Building Violations update process

We have upgraded the mechanism that updates the Building Violations dataset every day.  The main visible effect of this change is that the record counts for 2013-2015 have increased by about 80 percent.  We found some modifications and new building violations were not updating or being correctly added on the portal but are now doing a complete re-query of the source database and complete replace of the dataset each time we update.

The last version of the dataset before the change in update mechanism is visible at https://data.cityofchicago.org/id/f4ic-vjbw.  This link may expire at some point so if you need the records, we recommend extracting and saving them.