Session 3: Building Damage Analysis

Session 3: Building Damage Analysis#

You will learn about fragility curves and mappping, building inventory dataset, and buiding damage analysis by using IN-CORE in this session

3.1 Fragility Curve and Mapping#

Damage analyses use fragility curve sets to calculate the limit state and damage stsate. Mapping then determines which fragility curve set applying to which inventory. For example, a building that has certain number of stories could be mapped to a specific set of fragility curves.

In IN-CORE, A user can both upload a set of fragility curves and mapping to the DFR3 services, or use local curve and mapping set objects.

This section briefly shows how to access fragility curves. A detailed instrcution of how to create your own fragility entities and mapping entities can be found in our Mannual: Toutorial - Create DFR3 Object

User can easily browsing and searching Fragility Curves and Mappings via DFR3 Viewer dfr3_viewer

Analyses use DFR3 mapping sets from IN-CORE service to associate each element of a given set with one or more elements of a second set of DFR3 curves. In IN-CORE and pyIncore specifically, a building inventory (given set) is mapped to a (second) set of fragility curves.

Please note that in this notebook we use term function/method interchangably in programming context as a class function and curve as a function in mathematical context (lognormal curve).

Read fragility curve set#

# import pyIncore modules
from pyincore import FragilityService, IncoreClient, FragilityCurveSet, MappingSet

# Connect to IN-CORE service
client = IncoreClient()

# Initalize a FragilityService
fragility_services = FragilityService(client)

# Getting a fragility curve and metadata
# You can see the curve with this ID at DFR3 viewer
fragility_set_id = "5b47b2d7337d4a36187c61ce"

# Get local fragility curve set
fragility_curve_set = FragilityCurveSet(fragility_services.get_dfr3_set(fragility_set_id))
fragility_curve_set.fragility_curves

Visualize this fragility curve using pyincore-viz package#

from pyincore_viz.plotutil import PlotUtil as plot

plt = plot.get_fragility_plot(fragility_curve_set, title="Mid-Rise Steel Moment Frame")
plt.show()

3.2 Building Inventory#

The schema (columns) of building Inventory has been going through several iteration (v5, v6, v7), and here is a list of supported column names in version 6. Having a fully populated Building Inventory, with no additional columns, will hold all the data needed to perform all analyses that are compatible with that version of the Building Inventory.

column name	description	type
GUID	added by IN-CORE	string
ARCHETYPE	building structure archetype	integer
OCC_TYPE	Broad HAZUS Occupancy Category (e.g. RES3 - multi-family residential)	string
APPR_BLDG	Appraised value for the building	double
SQ_FOOT	total building area in square feet	integer
DWELL_UNIT	total number of dwelling units in the building	integer
NO_STORIES	total number of stories for the building	integer
CONT_VAL	value of building contents	double
STR_TYP2	detailed structure type as per HAZUS MR-3 specifications	string
EFACILITY	essential facility designation	string
PAR_ID	parcel identifier	string
PARID_CARD	improvement identifier	string
BLDG_ID	building identifier (unique)	string
STR_PROB	probability that the structure type specified is correct	double
YEAR_BUILT	the year the structure was built	integer
A_STORIES	the number of above ground stories	integer
B_STORIES	the number of below ground stories	integer
BSMT_TYPE	the basement type	string
GSQ_FOOT	total ground floor area of the building in square feet	integer
OCC_DETAIL	“specific occupancy category, describing the detailed use of the building”	string
MAJOR_OCC	major occupancy category for the parcel in which the building is sited	string
BROAD_OCC	general occupancy categories	string
REPL_CST	replacement cost for the building from R.S. means square foot cost	double
STR_CST	structural component of the replacement cost	double
NSTRA_CST	acceleration sensitive component of replacement cost	double
NSTRD_CST	drift sensitive component of replacement cost	double
DGN_LVL	design level for the building as per HAZUS MR-3 specifications	string
OCC_TYP2	detailed HAZUS occupancy category for the building	string
TRACT_ID	census tract identifier	string

There are many ways to create building inventory dataset; for example 1) using GIS software (e.g. ArcGIS, QGIS, etc), 2) using python code. In this exerercise, we will create a buiding inventory dataset in ESRI Shapefile from CSV file by using GeoPandas library.

GeoPandas is an open source project to make working with geospatial data in python easier. GeoPandas extends the datatypes used by pandas to allow spatial operations on geometric types. For more information, please follow the user guide

Create a dataset object#

Any analysis in pyIncore, by default uses Dataset Object as an input. Hence we have to convert building dataset into pyincore dataset object, in order to use it as an input for damage analysis. We can either upload the dataset to IN-CORE data services to acquire an id; or using existing dataset id from the services; or using local files.

Create building inventory dataset object from local files#

Below we will create a buidling inventory data in ESRI shapefile from CSV file according to building inventory schema. With the files, we will create a local dataset object.

# import Dataset class
from pyincore import Dataset

Step 1: Convert CSV file to ESRI Shapefile#

Open CSV file in the file browser to see the contents.

# Load CSV file into GeoPandas, then export the GeoDataFrame to ESRI shapefile

import pandas as pd
import geopandas as gpd

df = pd.read_csv('ergo_memphis_hospitals.csv')

gdf = gpd.GeoDataFrame(df, crs='epsg:4326', geometry=gpd.points_from_xy(df.lon, df.lat))
gdf.to_file("sample_bldgs.shp")

Step 2: Add GUID#

Each built-in infrastructure needs a GUID. A GUID is an acronyom that stands for Globally Unique Identifier, they are also referred to as UUIDs or Universaly Unique Identifiers. Technically they are 128-bit unique reference numbers used in computing which are highly unlikely to repeat when generated.

# pyincore has utility methods help user easily add GUID
from pyincore import GeoUtil

GeoUtil.add_guid("sample_bldgs.shp", "sample_bldgs_w_guid.shp")

Step 3: Create a local dataset for IN-CORE#

The following code create a local Dataset object with ESRI shapefile for buiding inventory

# use the method "from_file" in Dataset class to construct dataset object from local file
local_buildings = Dataset.from_file("sample_bldgs_w_guid.shp", data_type="ergo:buildingInventoryVer7")

(Optional) Create a dataset from IN-CORE Data Service#

A dataset can be also loaded from data service with an id. pyIncore has utility methods to download the files and metadata from the data service to create a dataset obejct in local.

(Optional) Upload local dataset to IN-CORE Data Service and get ID#

Write Metadata: Metadata is a string describing the dataset.
dataType needs to be align with the analyses in pyincore.
format is the file format of the dataset. Currently we support “shapefile”, “table”, “Network”, “textFiles”, “raster”, “geotiff” and etc. Please consult with development team if you intend to post a new format.

3.3 Configure the Building Damage Analysis#

This analysis computes building damage based on a particular hazard. In this secion, you will perform building damage anlayis for both scenario earthquake and tornado.

The process for computing the structural damage is similar to other parts of the built environment. First, a fragility is obtained based on the hazard type and attributes of the building. Based on the fragility, the hazard intensity at the location of the building is computed. Using this information, the probability of exceeding each limit state is computed, along with the probability of damage. For the case of an earthquake hazard, soil information can be used to modify the damage probabilities to include damage due to liquefaction.

The outputs of this analysis are CSV file with probabilities of damage and JSON file with information about hazard and fragilities. The detail information about the analysis in our manual: Building damage analysis

Input parameters

key name	type	name	description
`result_name` ^*	`str`	Result name	Name of the result dataset.
`hazard_type` ^*	`str`	Hazard type	Hazard type (earthquake, tsunami, tornado, hurricaneWindfields).
`hazard_id` ^*	`str`	Hazard id	ID of the hazard from the Hazard service.
`fragility_key`	`str`	Fragility key	Fragility key used in mapping dataset.
`use_liquefaction`	`bool`	Liquefaction	Use liquefaction, if applicable to the hazard. Default is False.
`use_hazard_uncertainty`	`bool`	Uncertainty	Use hazard uncertainty. Default is False.
`num_cpu`	`int`	Number of CPUs	Number of CPUs used for parallel computation. Default is 1.

Input datasets

key name	type	name	description
`buildings` ^*	`ergo:buildingInventoryVer4` `ergo:buildingInventoryVer5` `ergo:buildingInventoryVer6` `ergo:buildingInventoryVer7`	Building dataset	A building dataset.
`dfr3_mapping_set` ^*	`incore:dfr3MappingSet`	DFR3 Mapping Set	DFR3 Mapping Set.

Output datasets

key name	type	parent key	name	description
`ds_result` ^*	`ergo:buildingDamageVer5`	`buildings`	Results	A dataset containing results (format: CSV).
`damage_result` ^*	`incore:buildingDamageSupplement`	`buildings`	Results	Information about applied hazard value and fragility (format: JSON).

(* required)

from pyincore.analyses.buildingdamage import BuildingDamage

Run Damage with earthquake#

# Define the variables to be used for input parameters

hazard_type = "earthquake"
hazard_id = "5b902cb273c3371e1236b36b"

result_name = "memphis_eq_bldg_dmg_result"

# Default Building Fragility mapping
mapping_id = "5b47b350337d4a3629076f2c"

# create fragility mapping object with id
eq_mapping_set = MappingSet(fragility_services.get_mapping(mapping_id))

# Create building damage analysis object
eq_bldg_dmg = BuildingDamage(client)

# Load local dataset (building inventory)
eq_bldg_dmg.set_input_dataset("buildings", local_buildings)

# Load fragility mapping
eq_bldg_dmg.set_input_dataset("dfr3_mapping_set", eq_mapping_set)

eq_bldg_dmg.set_parameter("result_name", result_name)
eq_bldg_dmg.set_parameter("hazard_type", hazard_type)
eq_bldg_dmg.set_parameter("hazard_id", hazard_id)
eq_bldg_dmg.set_parameter("num_cpu", 1)

# Run building damage analysis
eq_bldg_dmg.run_analysis()

Take a glance of the output#

eq_bldg_dmg_table = eq_bldg_dmg.get_output_dataset("ds_result").get_dataframe_from_csv()
eq_bldg_dmg_table.head()

3.4 Post-process and Vizualize the Damage Analysis Output#

Joining dataset#

Data preparation and data post-processing are common procedures. Prior to using pyIncore, users often encounter situation that they need to reshape their own dataset to make it compliant with the input dataset format of pyIncore.

For example, adding GUID to inventory dataset. GUID (aka UUID) is an acronym for ‘Globally Unique Identifier’. It is a 128-bit integer number used to identify resources. In IN-CORE We require each individual row/item in an inventory to have its own GUID. If your dataset does not come with GUID, pyIncore has utility method to help you add it.

After acquiring outputs from pyIncore analyses, often time user would need to perform data aggregation to gain statitical insights. The below tutorial gives a few examples on how to join datasets and generate some basic visualizations.

# getting dataframe of damage output
eq_bldg_dmg_df = eq_bldg_dmg.get_output_dataset("ds_result").get_dataframe_from_csv()

# getting geodataframe of building dataset 
bldg_gdf = local_buildings.get_dataframe_from_shapefile()

# merge/join two dataframe
# you can choose columns to be merged
bldg_dmg_gdf = bldg_gdf[['guid', 'struct_typ', 'geometry']].merge(eq_bldg_dmg_df, on='guid')
bldg_dmg_gdf.head()

Display tabular data#

You can use Pandas and GeoPandas to dispaly tabular data and create charts. This secsion shows some examples.

Describe#

# Stat summary on a column
bldg_dmg_gdf["LS_0"].describe()

Sort#

# Show table sorted by DS_3 (decending) and struct_typ (ascending)
bldg_dmg_gdf.sort_values(['DS_3', 'struct_typ'], ascending=[0,1]).head()

Group by#

# Show table group by Structure type
grouped_bldg_dmg = bldg_dmg_gdf.groupby(by=['struct_typ'], as_index=True)\
.agg({'DS_0': 'mean', 'DS_1':'mean', 'DS_2': 'mean', 'DS_3': 'mean', 'guid': 'count'})
grouped_bldg_dmg.rename(columns={'guid': 'total_count'}, inplace=True)
grouped_bldg_dmg.head()

Simple barchart using pandas#

# Plot Damage state by structure type
ax = grouped_bldg_dmg[["DS_0", "DS_1", "DS_2", "DS_3"]].plot.barh(stacked=True)
ax.set_title("Stacked Bar Chart of Damage State Grouped by Structure Type", fontsize=12)
ax.set_xlabel("complete damage value", fontsize=12)
ax.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) #here is the magic

# Create a histogram with DS_3
ax = bldg_dmg_gdf['DS_3'].hist(bins=20, figsize=[10,5])
ax.set_title("complete damage distribution", fontsize=12)
ax.set_xlabel("complete damage value", fontsize=12)
ax.set_ylabel("counts", fontsize=12)

Using pyIncore-viz#

pyIncore-viz is a Python visualization package for pyIncore module. It provides users with convenient methods to create image and graphic outputs of various hazard IN-CORE analyses.

Plot Geospatial Map#

Plotting a geospatial map is generating static image of a map.

from pyincore_viz.geoutil import GeoUtil as geoviz

# Plot earthquake
eq_id = "5b902cb273c3371e1236b36b"
geoviz.plot_earthquake(eq_id, client)

# Plot a map with a Dataset 
# with local dataset (local_buildings)
geoviz.plot_map(local_buildings, column='struct_typ',category='True')

# Plot a map with GeoDataframe 

geoviz.plot_gdf_map(bldg_dmg_gdf, 'DS_3', basemap=False)

# Overlay GeoDataFrame with a base map
geoviz.plot_gdf_map(bldg_dmg_gdf, 'DS_3', basemap=True)

# Interactive map with Dataset
# addjust additional multiplier to increase the intensity of heatmap, e.g. multipler=1000 
map = geoviz.plot_heatmap_from_gdf(bldg_dmg_gdf, "DS_3", radius=7, blur=5, name="DS_3", multiplier=1000) 
map