Skip To Content

Append Data

Append DataThe Append Data tool allows you to append features to an existing hosted layer in your ArcGIS Enterprise organization. Append Data allows you to update or modify existing datasets.

Workflow diagram

Append Data workflow diagram

Analysis using GeoAnalytics Tools

Analysis using GeoAnalytics Tools is run using distributed processing across multiple ArcGIS GeoAnalytics Server machines and cores. GeoAnalytics Tools and standard feature analysis tools in ArcGIS Enterprise have different parameters and capabilities. To learn more about these differences, see Feature analysis tool differences.

Examples

A large chain restaurant collects monthly sales records for each of its locations. To avoid maintaining datasets for each location each month, the company wants to use one annual sales layer for each location. The Append Data tool enables it to append data to a master dataset at the end of each month when its newly collected sales records are available.

Ecological Marine Unit studies are being conducted to better understand ocean floor patterns of the Atlantic Ocean over time. The studies are conducted using data gathered from various environmental organizations that have collected information from the Atlantic Ocean in the past. Because each organization maintains its own datasets, the Append Data tool can be leveraged to append multiple ocean floor datasets into one layer.

Analysis using GeoAnalytics Tools

Analysis using GeoAnalytics Tools is run using distributed processing across multiple ArcGIS GeoAnalytics Server machines and cores. GeoAnalytics Tools and standard feature analysis tools in ArcGIS Enterprise have different parameters and capabilities. To learn more about these differences, see Feature analysis tool differences.

Usage notes

The input layer is defined using the Choose layer to append to drop down. The input layer must be a hosted point, line, area, or table feature layer.

The append layer is defined using the Choose layer to append drop down. The append layer can be a point, line, area, or table big data file share dataset or feature layer.

It is required that the input layer and append layer have the same geometry type.

If time is enabled on the input layer, the two layers must have the same time type. To learn more about feature layer time settings, see Configure time settings. To learn more about big data file share time settings, see Time in Understanding a big data file share manifest.

You can optionally use the Append all features or define a subset filter by clicking the Query button Query. Only features that match the condition will be appended. For example, if you have a field named temperature, you could append all features with temperature values greater than 10 with the condition temperature is greater than 10.

The Choose how to append fields field mapping table appears after the input layer and append layer are selected. It will automatically populate with input layer fields and their matching values from the append layer.

By default, input layer fields will be appended with null values when they do not have a matching field in the append layer. Optionally, you can use the Choose how to append fields field mapping table to append custom values of the following two types:

  • Append Field—Match an input layer field with an append layer field of a different name but the same type.
  • Expression—Calculate input layer field values for append features. To learn more about using Arcade expressions and Append Data see, Arcade expressions with Append Data.

For example, human migration researchers want to append datasets with the input layer and append layer schemas shown below. Both layers have a field in common named Country of type text and both have two additional fields with the same data type, but unique names. The input layer has Pop_ and Pop_Density fields, and the append layer has Population and area_km2 fields. The researchers want to match the Country field to the Country field, append the Population field to the Pop_ field, and calculate the population density for the Pop_Density field using a calculation.

Append Data input layer and append layer schemas
Input layer and append layer example schemas to be used in the Append Data tool

By default, the field mapping table will match Country fields based on the field name and field type. The Pop_ and Pop_Density fields have no match in the append layer and will be appended with null values.

Append Data default field mapping
Default field mapping for the Choose how to append fields table

Use the Choose how to append fields field mapping table to match the input layer Pop_ field with the append layer Population field by selecting Population from the drop down list next to Pop_ under Append Value. Use the Expression option from the drop down list next to Pop_Density to calculate values for the appending features by using the append layer fields Population and area_km2 to build the Arcade expression $feature["Population"]/$feature["area_km2"].

Append Data defined field mapping
Defined field mapping for the Choose how to append fields table

The fields of the input layer are never modified. Any append layer fields that are not matched in the Choose how to append fields field mapping table will be excluded from the appended results.

Limitations

  • The features you append must have the same geometry type as the feature layer you append to.
  • The layer you append to must be an existing layer on your hosting server. If you want to append to a different layer, you must create a hosted layer of that dataset first. To do this, use the Copy To Data Store tool, or share a layer to your portal. To learn more about sharing layers, see Introduction to sharing web layers. Once your dataset is a hosted layer, you can run the Append Data tool to append features to it.
  • The Choose how to append fields field mapping table does not allow fields to be added, removed, or renamed in the input layer.

ArcGIS API for Python example

The Append Data tool is available through ArcGIS API for Python.

This example appends a big data file share of earthquakes in the Atlantic ocean to a feature layer of earthquakes.

# Import the required ArcGIS API for Python modules
import arcgis
from arcgis.gis import GIS
from arcgis.geoanalytics import manage_data

# Connect to your ArcGIS Enterprise portal and check that GeoAnalytics is supported
portal = GIS("https://myportal.domain.com/portal", "gis_publisher", "my_password", verify_cert=False)
if not portal.geoanalytics.is_supported():
    print("Quitting, GeoAnalytics is not supported")
    exit(1)   

# Find the big data file share dataset you're interested in using for analysis
search_result = portal.content.search("", "Big Data File Share")

# Look through search results for a big data file share with the matching name
bd_file = next(x for x in search_result if x.title == "bigDataFileShares_NaturalDisaters")

# Look through the big data file share for Earthquakes_atlantic
eq_atlantic = next(x for x in bd_file.layers if x.properties.name == "Earthquakes_atlantic")

# Find a feature layer named "Earthquakes" in your ArcGIS Enterprise portal
earthquakes = portal.content.search("Earthquakes", "Feature Layer")
earthquakes_layer = layer_result[0].layers[0]

# Set the tool environment settings 
arcgis.env.process_spatial_reference = 32618
arcgis.env.verbose = False

append_data_result = manage_data.append_data(earthquakes_layer, eq_atlantic) 

# Visualize the tool results if you are running Python in a Jupyter Notebook
processed_map = portal.map('World', 1)
processed_map.add_layer(append_data_result)

Similar tools

Use the ArcGIS GeoAnalytics Server Append Data tool to append features to layers on your hosting server. Other tools may be useful in solving similar but slightly different problems.

Map Viewer analysis tools

Select and copy data to a new feature layer in your portal using the ArcGIS GeoAnalytics Server Copy To Data Store tool.

Calculate values for features in a new or existing field using the ArcGIS GeoAnalytics Server Calculate Field tool.

ArcGIS Desktop analysis tools

To run this tool from ArcGIS Pro, your active portal must be Enterprise 10.6.1 or later. You must sign in using an account that has privileges to perform GeoAnalytics Feature Analysis.

Perform similar append operations in ArcGIS Pro with the Append geoprocessing tool.