Enterprise GIS and ArcGIS Online
Topics & Learning Objectives:
| Section | Learning Objectives |
|---|---|
| 1. Desktop GIS vs AGOL vs Enterprise GIS | • Describe key differences between desktop GIS, AGOL, and enterprise GIS • List a few alternatives to ESRI’s cloud GIS platforms |
| 2. ArcGIS Online as a Content Management System | • Describe ArcGIS Online in terms of a content management system • Add and document data to your ArcGIS Online account via the web interface • Identify the main components of metadata in AGOL • Differentiate among the different service types in AGOL • Share AGOL resources with groups, organizations, or globally • Describe the analytical capabilities of AGOL • Explain what “data enrichment” is • List and explain some AGOL apps and what functionality they add |
| 3. Geospatial Analysis in ArcGIS Online (optional) | • Describe the analytical capabilities of AGOL • Explain what “data enrichment” is • List and explain some AGOL apps and what functionality they add |
| 4. ArcGIS Online Apps | • List and explain some AGOL apps and what functionality they add |
1. Desktop GIS - vs - ArcGIS Online - vs - Enterprise GIS
Desktop GIS
So far, the vast majority of our learning GIS at NSOE has been done using “desktop GIS”, specifically ArcGIS Pro. What does this imply? Traditionally this means:
- All the data used in your analysis are stored on your desktop i.e., the machine in front of you (or a mapped drive).
- All processing is done on your desktop (or a single virtual machine)
- To share your data, analysis, and/or results, you need to copy or move files from machine to machine.
So, with desktop GIS, you need to supply your own hardware and your own data. You license the software to run analyses and manage data, and licenses are usually provided on a per-machine basis.
ArcGIS Online
ArcGIS Online, as its name implies, is a an on-line resource typically accessed via a web browser. You can search and browse data, create maps, and even perform analyses without needing any specialized software or data stored on your local machine. Thus, in contrast to desktop GIS:
- Data are stored and managed “in the cloud”.
- All processing of the data is done on remote servers.
- Sharing data and products are done via web links.
Resources are all remote in the AGOL platform, you don’t need a high-end machine and many datasets are accessible as on-line resources. Here you often pay for a subscription to access the resource. With ArcGIS Online, you will additionally pay for credits that count toward storing your own data or executing particularly intensive processing tasks, both of which consume server resources.
Enterprise GIS
Enterprise GIS is a bit of a hybrid between desktop GIS and ArcGIS Online. From a user’s perspective, Enterprise GIS appears quite similar to ArcGIS Online. The key difference is that, instead of using ESRI’s hardware to host data and run centralized processes, you set up an ArcGIS online portal on your own servers. The servers used to host the portal are usually higher end machines than your typical desktop computers, costing a bit more and often requiring dedicated management (ensuring they are always available and backed up).
With enterprise GIS, you again need to purchase software and also budget management of the server, and of course you have the upfront cost of the server equipment. But you don’t need subscription fees or credit accounting as all the heavy lifting is done on your own equipment.
ESRI and Non-ESRI Options for Enterprise GIS
ESRI is the big name in all of this, but not the only company offering Enterprise GIS solutions.
GeoServer (http://geoserver.org/) is an open source alternative to hosting data on servers or a cluster of servers. It provides both viewing and access to geospatial data via web services, as AGOL does. Admittedly, I have limited experience with GeoServer. With all open source software, you get the benefit of having a much larger body of programmers improving and expanding the platform, but it comes at the cost of uncertain stability and minimal documentation and no dedicated support. Yes, it’s free, but your time is not, and if you go this route, it’s best to understand that you will be likely spending significantly more time setting it up and managing it.
With data storage, management, and distribution a key component in enterprise GIS, many commercial and open source database management systems, or DBMS, can also provide enterprise GIS services. Oracle and Microsoft SQL Server are the giants in commercial DMBS, providing proven and robust products, but at a premium price. (Though Microsoft often has generous agreements with education and non-profit communities, as does ESRI for that matter.) PostgreSQL seems to be the leading open source DMBS. Some smaller end DMBS packages are MySQL and SQLite, both of which are open source, but are note designed for enterprise scale database management.
In addition to standard, online access points to large relational databases, these DBMS also have plugins or adaptors that allow you to include geometric features in their databases. Additionally, some also include mechanisms to perform spatial analysis. I again have very limited experience with GIS data in these formats; typically, if you are going this route you are less likely an aspiring environmental manager and more someone with a dedicated technology background. (They involve quite a bit of computer savvy!) Still, these technologies are useful to know, even as – or perhaps especially as –environmental managers.
Summary
If you are ever in the position of setting up your own shop, know that you have options for how you store your GIS data and run analyses. The options sketched out above each have different costs layouts (in $ and time) and expandability. Up until recently, for shops the up to the scale of a small to mid-size NGO, my recommendation was to invest in one or a few high end desktops and go the desktop GIS route. Why? Because it’s proven to be capable and stable, doesn’t require too much computer support know-how, and has a rich user community. However, I’m now starting to embrace ESRI’s ArcGIS Online platform for reasons I’ll layout in detail next…
Getting to know ArcGIS Online
After spending much time with ArcGIS Pro (i.e. desktop GIS), let’s explore ArcGIS Online, taking note of what roles it plays in the overall “GIS Ecosystem”. Before we embark, however, it’s important to acknowledge that ArcGIS Online is a rapidly changing platform, even more so that ArcGIS Pro. As such, it’s perhaps best to concentrate on the overall structure and functionality it serves rather than the exact details of everything.
2. ArcGIS Online as a Content Management System
A primary role of AGOL is to store, document, and share data which can be used in geospatial analysis, i.e. as a content management system. We can upload tables, shapefiles, geodatabases, and a few other file formats into our AGOL accounts (Organizational or Enterprise) which then become accessible to download from anywhere. Additionally, we can convert these uploaded datasets into web services so that we or others can use the data directly without needing to download them.
One key feature of AGOL as a content management system is its ability to set access to datasets on a granular (or per-dataset) level. Every resource stored on AGOL can be restricted to just the owner of the dataset, to one or more defined groups of users, or to entire organizations. Or data can shared without restriction, open for all to view and use.
And finally, each dataset can be fully documented with metadata. Besides just its filename, each dataset can be associated with a summary, a description, searchable tags, categorical tags, usage restrictions/licensing, and credits. For some datasets, you can also set the defaults for how the data should be displayed. And finally, all datasets have an option of being tagged as authoritative, if has been peer reviewed, or deprecated, if support or updating of the dataset will imminently be discontinued.
The exercise below is designed to give you a practical overview of how data is uploaded, documented, and shared on ArcGIS Online.
♦ Exercise 1: Publishing Data on ArcGIS Online
Download the data here for this example…
- Upload a csv file
- Upload a shapefile
- Upload a geodatabase
- Document a file
- Share a file
3. Geospatial Analysis in ArcGIS Online
Until fairly recently, AGOL’s “analytical” capability was limited to creating and sharing web maps where users could zoom, pan, symbolize, and highlight features in vector datasets. Raster data could be added to maps as well, but really only for viewing, not analysis. In the last few years, however, ESRI has introduced some fairly powerful analytical capability in ArcGIS Online.
https://www.esri.com/en-us/arcgis/products/arcgis-online/capabilities/analyze-data
From the above, you’ll note that in addition to some fundamental geospatial analyses (including summarizing data, buffering, and overlaying data) , ESRI now includes some quite powerful analytical capabilities to its AGOL platform. These include:
- Some rich spatial statistical analyses for exploring patterns in your data. These are quite useful when you have “big” datasets, meaning you have an abundance of data and need to sort through it to see what can be gleaned from it. See this link for more information.
- Data enrichment, which allows you to append to your existing data a rich set of demographic, business, and economic data curated by ESRI. See this link for more information.
- Network analysis using a built-in street network dataset, which allows you to compute drive-time or walk-time distances from features, calculate shortest routes, and service areas from spatial features.
- Terrain and Hydrologic analyses that enable you to trace downstream paths or upstream areas from features or compute viewsheds from a set of locations.
- Geocoding tools enable you to translate back and forth between addresses and geographic coordinates using ESRI’s authoritative address locators.
A note on credits:
The geospatial analyses mentioned above all consume credits. This is because these analyses, some of them which can be quite significant, run on ESRI’s CPUs which need to be spun up and dedicated to your request. ESRI has generously provided the Nicholas School with abundant credits so we don’t need to be concerned with this, but outside of Duke you’ll likely need to economize your use.
Each analysis tool usually has an option to preview the credits that an analysis will consume…
♦ Exercise 2: Executing Analysis via AGOL’s MapViewer
Here we’ll leverage ESRI’s extensive tutorial collection to delve into the analytical capabilities of ArcGIS online.
- Add the EV Charger Point locations to your map
- Buffer the features 2.5 km
- How many credits did that consume?
- Zoom to Durham (search for Durham in the search box)
- Compute 10 minute drive time areas from each site within the extent of the map.
- How many credits did that operation consume?
4. ArcGIS Online Apps
In addition to hosting data, allowing users to create and share maps, and analyzing data, ArcGIS Online also supports a number of add-on applications it supports. The full list of add-on applications can be found by clicking the 
icon just to the left of your profile name in the upper right corner of your AGOL home page. When you do you should see all the apps (though some may be hidden at first):
We will explore a few of these - Insights, Dashboards, and the Experience Builder in a bit more detail in separate exercises.
- Dashboards Tutorial: Build an Interactive Dashboard
- Experience Builder Tutorial: Getting Started with ArcGIS Experience Builder
- Insights Tutorial: Map racial disparities in breast cancer mortality rates
Summary
ArcGIS Online has come a long way in very little time over the last few years. Certainly ESRI is pouring many resources into this product, and for good reason! Connectivity among devices is becoming more ubiquitous and more reliable across the world. That, combined with the fact that we now use a spectrum of computing devices (“internet of things” appliances, phones, tablets, notebooks, desktops, servers, server farms…) means that data and processing are more and more distributed. In other words, the desktop - which held the throne as the workhorse of getting things done - is now limiting if used in isolation!
From here, we’ll dig a bit deeper into the mechanics of all this. In the next lesson, we explore cloud computing from the data perspective. We see how data stored in ArcGIS online can be accessed outside of ArcGIS Online in various formats. And in doing so, we discuss what’s going on “under the hood” - which empowers us to tap into more data and other cloud resources!