Help us to build a collaborative community for the future development of Open Government and Open Data Initiative for San Diego!

Why sponsor?

          The Big Data Hackathon for San Diego aims to promote the development of data science and information technology solutions for San Diego on important civic issues related to water conservation, disaster response, and crime monitoring. Sponsorship of this important hackathon event will give you access to top talent and emerging stars in the information technology, data science and computer science fields.

          Over 200 participants are expected to attend this free event. Sponsorship can give you an opportunity to build company and brand recognition, to mentor and judge the innovative work created at the Hackathon, and to present your recent big data efforts to a large and targeted audience. If you are interested in becoming a sponsor, please download our flyer to see sponsorship levels or contact Professor Amy Schmitz Weiss about sponsorship opportunities at aschmitz@mail.sdsu.edu.

• License query system hosted by the California Dept. of Alcoholic Beverage Control. Users of the system can generate reports on daily, weekly, and monthly time frame. Ad-hoc reports are also optional in the system.

• Dataset
  • License Query System - Reports

• Processed crime incidents, based on data supplied by SANDAG.

  Important:

  • See Caveats for limitations and warnings regarding this data.
  • Use of this data is subject to multiple terms and conditions. See Terms for details.

  This dataset includes geocoded crime incidents from 1 Jan 2007 to 31 March 2013 that were returned by SANDAG for Public Records request 12-075.

  The extracts from the dataset include both CSV files, for use in spreadsheet applications, and ESRI shapefiles, for use in GIS applications. Both files are partitioned by year and include the following fields. Shapefiles also include point geometries for each incident.

  Many of the columns use Clarinova Place Codes (CPC) to identify places, such as neighborhoods, council districts and communities.

• Dataset
  • San Diego Region Crime Incidents 2007 – 2013

• Live Feeds hosted by Esri as REST Service (including earthquake and wildfire)

• Dataset
  • ArcGIS Live Feeds

• Fire, evacuation, and other hazard boundaries are approximate and can change rapidly. This map is based on official mapping sources that show rough footprints of active emergencies and emergency notification areas from responding agencies that may take several hours to produce. Zooming into specific locations may produce misleading information. Please monitor official County of San Diego updates at www.sdcountyemergency.com/updates or call 2-1-1 for specific emergency information such as road closures and evacuations.

• Dataset
  • San Diego County Emergency Data Feed (ArcGIS Feature Service)

• USGS Data Service is a portal that collects all the REST endpoint that are hosted by USGS. Background Map, Base Data Layers, Natural HAzard, Web Feature Services(WFS), and other data are provided.

• Dataset
  • USGS Data Service

• NCDC's Climate Data Online (CDO) offers web services that provide access to current data. This API is for developers looking to create their own scripts or programs that use the CDO database of weather and climate data. An access token is required to use the API, and each token will be limited to five requests per second and 1,000 requests per day by default.

• Dataset
  • NOAA CDO Web Services

• This page shows alerts currently in effect for San Diego (CAC073) California and is normally updated every two-three minutes.

• Dataset
  • NOAA CAP(Common Alerting Protocol)

• FEMA offers several ways for you to receive automatic updates on press releases, disaster declarations, our latest blog posts, and information from our partners. The data feeds page is intended to be a one stop shop for FEMA's open data. This data is intended for use by our whole community stakeholders and you can find more information about the metadata within each dataset. Click on the icons next to the sources below to open the data in your preferred format.

  Data Feeds include APIs, Disaster Declarations, Datasets in tables, and GIS Data

• Dataset
  • FEMA Data Feeds

• The Climate Prediction Center's (CPC) products are operational predictions of climate variability, real-time monitoring of climate and the required data bases, and assessments of the origins of major climate anomalies. The products cover time scales from a week to seasons, extending into the future as far as technically feasible, and cover the land, the ocean, and the atmosphere, extending into the stratosphere.

  These climate services are available for users in government, the public and private industry, both in this country and abroad. Applications include the mitigation of weather related natural disasters and uses for social and economic good in agriculture, energy, transportation, water resources, and health. Continual product improvements are supported through diagnostic research, increasing use of models, and interactions with user groups.

• Dataset
  • NOAA Climate Prediction Center - GIS Data

• The State of California is experiencing one of the most severe droughts on record, which has implications for citizens of California and beyond. Many State, Federal, and Tribal agencies make routine observations of the water cycle. This website graphically visualizes these data to help understand the effect of drought on rivers, streams, lakes, and reservoirs. Below, learn more about water use and changes in the water cycle due to the current drought in California.

  The data presented on the "California drought, visualized with open data" are drawn from free and publicly accessible sources. In addition, the analytical, graphical, and software tools used are open-source and available for public re-use.

• Dataset
  • Github site for the CIDA drought data map

• The USGS's National Water Use Information Program compiles and publishes the Nation's water-use data. Public access to some of these data is provided via the USGS Water Data for the Nation site (additional background). Water use refers to water that is used for specific purposes. Water-use data is collected by area type (State, county, watershed or aquifer) and source such as rivers or groundwater, and category such as public supply or irrigation. Water-use data has been reported every five years since 1950, for years ending in "0" and "5". The USGS works in cooperation with local, State, and Federal agencies as well as academic and private organizations to collect and report total withdrawals.

  Annual water-use data are expressed in units of million gallons per day (abbreviated as Mgal/d) and thousand acre-feet per year. For the irrigation category, water-use data are the average daily quantities used over a period of a year and do not represent actual daily rates. In most parts of the nation, irrigation water is applied during only a part of each year and at variable rates; therefore, the actual rate of application is much greater than the average daily rate. For all other categories, the daily quantity of water that is withdrawn may be based on daily usage or more often, monthly usage that is averaged to a daily quantity.

• Dataset
  • California Water Use Data

• This page is a portal to all kinds of information on the current and past state of water resources, both in California and around the country. Get real-time information on California water resources via a searchable Google map interface.

• Dataset
  • USGS California Water Data

• The U.S. Drought Monitor, established in 1999, is a weekly map of drought conditions that is produced jointly by the National Oceanic and Atmospheric Administration, the U.S. Department of Agriculture, and the National Drought Mitigation Center (NDMC) at the University of Nebraska-Lincoln. The U.S. Drought Monitor website is hosted and maintained by the NDMC.

  U.S. Drought Monitor maps come out every Thursday morning at 8:30 Eastern Time, based on data through 7 a.m. Eastern Standard Time (8 a.m. Eastern Daylight Time) the preceding Tuesday. The map is based on measurements of climatic, hydrologic and soil conditions as well as reported impacts and observations from more than 350 contributors around the country. Eleven climatologists from the partner organizations take turns serving as the lead author each week. The authors examine all the data and use their best judgment to reconcile any differences in what different sources are saying.

• Dataset
  • UCDM GIS Data Archive

• The California Health and Human Services Agency (CHHS) has launched its Open Data Portal initiative in order to increase public access to one of the State’s most valuable assets – non-confidential health and human services data. Its goals are to spark innovation, promote research and economic opportunities, engage public participation in government, increase transparency, and inform decision-making.

  "Open Data" describes data that are freely available, machine-readable, and formatted according to national technical standards to facilitate visibility and reuse of published data.

• Dataset
  • CHHS Open Data Portal

• The San Diego Regional Data Library is dedicated to helping non profits, civic organizations, journalists and governments in the San Diego region get data they need to tell their stories and run their operations. This site, data.sandiegodata.org, is the main data repository for the Library. It holds unprocessed data sets received from other organizations, as well as data that the Library has processed to make them easier to use.

  All of the data in the library is subject to the Library's Data Disclaimers and Limitations, and data that originated with other organizations will be subject to the originators legal terms.

• Dataset
  • San Diego Regional Data Library

• The Regional GIS Data Warehouse provides data for everything from addresses to zoning, including: roads, property, parks, lakes, topography, census, and dozens of other layers - over 270 layers in 29 different categories. The public facing data are free to download after accepting the data disclaimer which is presented to each user upon entering the Regional GIS Data Warehouse.

  The data are stored as compressed "Shape" files (SHP). Metadata (information about the data) are provided in PDF format. A complete list of available layers can be downloaded as an Excel or PDF file.

• Dataset
  • Regional GIS Data Warehouse

• Your go-to data warehouse for the San Diego region. Get accurate census, estimate and forecast information and reports based on your specific needs—all in just a few clicks.
• Dataset
  • Data Surfer

• Public Crime Data in San Diego is made available by the Automated Regional Justice Information System (ARJIS). ARJIS was created as a joint powers agency to share information among justice agencies throughout San Diego and Imperial Counties, California.
• ARJIS Homepage
• Dataset
  • Public Crime Data Extract
  • Regional Crime Mapping System and Subscription Service

• R is a free software environment for statistical computing and graphics
• Homepage
• Tutorial
  • R tutorial
  • DataCamp
• Video
  • R Tutorial: Introduction to R
  • Introduction to R for Data Mining
• Books
  • An Introduction to Statistical Learning with Applications in R
• IDE
  • RStudio
• Useful links
  • R and Data Mining
  • The Open Source Data Science Masters

• Python is powerful… and fast; plays well with others; runs everywhere; is friendly & easy to learn; is open
• Homepage
• Python Documentation
  • Installation guide, tutorial, and some other tips
• Python Wiki
  • Beginners guide, list of books, learning materials, etc.
• Python Tutorial
  • Learn Python (Interactive)
  • Codecademy (Interactive)
  • Google’s Python class
• Useful Python Packages
  • Basic: numpy, scipy
  • Data Mining, data analysis, deep learning: scikit-learn, Pandas, Theano
  • Machine Learning: mlpy, PyBrain
  • Natural Language Toolkit: NLTK
  • Network: NetworkX, graph-tool
  • Webscraping: Scrapy, Beautiful Soup
  • Data Plotting: Orange, matplotlib
  • Interactive Development: IPython
  • Cross platform app development: PyQt, kivy
• Video
  • Python tutorials
  • Intro to scikit-learn
• Free Python books
  • Dive Into Python
  • Think Python
  • WIKIBOOKS: Python Programming
  • WIKIBOOKS: Python Programming (Non-programmer 2.6)
  • WIKIBOOKS: Python Programming (Non-programmer 3)
  • Pandas Cookbook

• A program language most commonly used to create dynamic web browsers
• JavaScript Tutorial
  • Learn JS (Interactive)
  • Codecademy (Interactive)
  • w3schools
• Cross-platform runtime environment
  • Node.js Tutorial
• JavaScript Libraries
  • Framework: jQuery, Dojo, AngularJS, backbone.js, Require.JS
  • UI: jQuery UI, Bootstrap
  • Data Visualization: d3, dygraphs, dataTable, morris.js
  • Time: moment.js
  • Map: OpenLayers API, Leaflet API, Google Maps API, ESRI ArcGIS JavaScript API, heatmap.js, Leaflet markerCluster

• Twitter
  • Overview
  • Tutorials: tuts
  • Tutorials: Codecademy
• Facebook
  • Overview
  • Tutorials: tuts
• Flickr
  • Overview
  • Tutorials: tuts
• Instagram
  • Overview
  • Tutorials: tuts
• Foursquare
  • Overview

• ArcGIS
  • Overview
  • Tutorial
• Google Map
  • Overview
  • Tutorials: w3schools
• Bing Map
  • Overview
  • Tutorials
• Yahoo! Maps
  • Overview
• OpenLayers
  • Overview
• USGS GAP
  • Overview

• Andriod
  • Overview
• Apple
  • Overview
  • Tutorials: Swift
  • Tutorials: Xcode
• Windows
  • Universal Windows Platform apps for Windows 10
• Cross platform app development
  • Qt
  • Corona
  • Ionic
  • AppStudio (ESRI)

• HTML5
  • Tutorials: w3schools
  • Tutorials: Udemy
  • Tutorials: Tutorials Point
• CSS3
  • Tutorials: w3schools
  • Tutorials: tuts
  • Tutorials: CSS3
• Ruby on Rails
  • Overview
  • Tutorials: Rails Tutorial
  • Tutorials: Ruby on Rails
  • Tutorials: Codecademy

• Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.
• Homepage
• Tutorial
  • Setting up with Amazon EC2
  • Getting Started with Amazon EC2
  • How to Setup Amazon Web Services EC2 Instance with Apache, PHP, MySQL
  • Amazon EC2 Setup Tutorial
  • STEP BY STEP AWS EC2 TUTORIAL

• Focus on creating amazing user experiences and forget complex infrastructure. Instantly add a powerful cloud database, push notification services, and analytics tracking to your app. Choose from over thirteen native SDKs for mobile, desktop, and IoT devices and discover all that Parse has to offer.
• Homepage
• Tutorial
  • Parse Tutorial
  • iOS and Swift Beginner Tutorial: App like Twitter using Parse Part 1
  • Getting Started with Parse
  • Adding a Cloud Backend for Your iOS App Using Parse – Part 1
  • Creating a Cloud Backend for Your Android App Using Parse

• ArcGIS Online is an online, collaborative web GIS that allows you to use, create, and share maps, scenes, apps, layers, analytics, and data. You get access to ready-to-use maps, ready-to-use apps, and Esri’s secure cloud, where you can add items and publish web layers.
• Homepage
• Tutorial
  • Videos
  • Get started with ArcGIS Online
  • ArcGIS Online Learning & Help Resources
  • ArcGIS Online Tutorials

• Dropbox is a home for all your photos, docs, videos, and files. Anything you add to Dropbox will automatically show up on all your computers, phones and even the Dropbox website — so you can access your stuff from anywhere.
• Homepage
• Tutorial
  • Dropbox Tour
  • Dropbox API

• GitHub is a Web-based Git repository hosting service, which offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features.
• Homepage
• Tutorial
  • Github Help
  • Set up a Git
  • Create a repository
  • Fork a repository
  • Github Desktop
  • Getting Started with GitHub Desktop
  • Contributing to Projects with GitHub Desktop

• How can Californians know about their water intake and how to conserve during the drought?
• Which day (weekday or weekend) or which areas (zipcode or street block) are the highest water usage?
• What are the reasons that people do not follow the mandatory water use restrictions in San Diego? http://www.sandiego.gov/water/conservation/drought/prohibitions.shtml
• Where are the citations of water use violation in San Diego? http://www.sandiego.gov/water/conservation/drought/enforcement.shtml

• In what ways can Californians be informed and be able to mobilize when the next disaster happens?
• How can we provide and share useful information for our local community during and after the disaster events (wildfires, floods, and earthquakes)? (like Craigslist services?)
• Where are the available shelters or local assistance center (LAC) for my family during disaster events? How long will it take for my family to move to the nearby shelters during the disaster?

• How can Californians know about the various crimes in their neighborhoods and how to curate and share this information?
• What are the major crime activities in my neighborhood recently (last two weeks)?
• Compared to last month or last year, will my neighborhood crime rate increase or decrease?
• Is it safe to go out in the late evening in my areas? When did the most crime activities happen in my area?