Pipeline Scrapes#

This table provides data about the daily updates on various aspects of pipeline activity, focusing on gas production. It offers detailed information about pipeline production in various regions, with a focus on specific locations and pipeline entities.

This is particularly useful for tracking energy production, monitoring pipeline activities, and analyzing regional energy trends.

To refer to the Pipeline Scrapes API endpoint, click here.

Columns#

Column Name	Type	Description	Example Value
DATE	TIMESTAMP_NTZ	Date of observation.	2024-05-30T00:00:00.000Z
ID	NUMBER	A SynMax assigned unique ID for each flow point.	30502
PIPELINE_NAME	VARCHAR	Name of pipeline as provided by FERC.	“TENNESSEE GAS PIPELINE”
LOCATION_NAME	VARCHAR	Name of the flow point as provided by the pipeline’s information posting disclosure.	“CPTENTEX/TGP COFFEEVILLE MS YALOBUS”
GAS_DAILY	FLOAT	Average rate of gas production expressed per day (in BCF/day).	12
REC_DEL	FLOAT	Represents Receipt or Delivery Point. 1 represents a Receipt Point, i.e. gas flowing into the pipe, whereas -1 represents Delivery Point, i.e. gas flowing out of the pipe. 0 represents data that is unavailable.	-1
COUNTY	VARCHAR	County in the state where the well is located.	“Yalobusha (MS)”
STATE_AB	VARCHAR	Abbreviation of the state.	“MS”
REGION	VARCHAR	Aggregation of multiple major producing basins.	“southeast”
SUB_REGION	VARCHAR	Aggregation of several counties, typically representing a major producing basin.	“MS”
CATEGORY	VARCHAR	Limit search to a type of flow point as described by the pipelines informational posting location data disclosure.	“LDC”
SOURCE	VARCHAR	Indicates the origin or method of data collection for the pipeline information provided by the API. Returned values are either “scrape” for webscraped values, “ffill” for forward filled values.	“ID3”
CREATED_DATETIME	TIMESTAMP_NTZ	Timestamp of when the record was created in the database.	2024-05-30T21:04:59.152Z

Example Data#

DATE	ID	PIPELINE_NAME	LOCATION_NAME	GAS_DAILY	REC_DEL	COUNTY	STATE_AB	REGION	SUB_REGION	CATEGORY	SOURCE	CREATED_DATETIME
2024-05-29T00:00:00.000Z	3350	Southern Star Central Gas Pipeline, Inc.	ETC - Rose Valley	41741	1	WOODS (OK)	OK	midwest	OK	Production	ID3	2024-05-30T21:04:59.152Z
2024-05-30T00:00:00.000Z	30502	TENNESSEE GAS PIPELINE	CPTENTEX/TGP COFFEEVILLE MS YALOBUS	12	-1	Yalobusha (MS)	MS	southeast	MS	LDC	ID3	2024-05-30T21:04:59.152Z
2024-05-29T00:00:00.000Z	29413	Northwest Pipeline LLC	LAPLATA-TW	0	-1	La Plata (CO)	CO	west	SanJuan-CO	Interconnect	ID3	2024-05-30T21:04:59.152Z
2024-05-30T00:00:00.000Z	31315	WBI Energy Transmission, Inc.	MDU-St. Marie Border	67	-1	Valley (MT)	MT	west	Montana	LDC	ID2	2024-05-30T21:04:59.152Z
2024-05-29T00:00:00.000Z	22905	Columbia Gas Transmission, LLC	ERVIN MORRIS UNIT #1-H1 $ 735044	4	1	GUERNSEY (OH)	OH	northeast	OH	Production	ID3	2024-05-30T21:04:59.152Z

Usage#

In Snowflake, you can write SQL queries to return the data you want.

Sample queries#

⮞ Return the total number of GAS_DAILY production for Region “midwest”

SELECT
  SUM(smx_pipes.gas_daily)
FROM
  smx_pipes
WHERE
  smx_pipes.region ILIKE '%midwest%';

Output

	SUM(SMX_PIPES.GAS_DAILY)
1	14521328591

⮞ Return all the records belong to the pipeline “Southern Star Central Gas Pipeline, Inc.”

SELECT
  *
FROM
  smx_pipes
WHERE
  smx_pipes.pipeline_name ILIKE '%Southern Star Central Gas Pipeline, Inc.%';

Output

DATE	ID	PIPELINE_NAME	LOCATION_NAME	GAS_DAILY	REC_DEL	COUNTY	STATE_AB	REGION	SUB_REGION	CATEGORY	SOURCE	CREATED_DATETIME
2024-05-29T00:00:00.000Z	30371	Southern Star Central Gas Pipeline, Inc.	Walton	0	-1	Marion (KS)	KS	midwest	KS	LDC	ID3	2024-05-30T21:04:59.152Z
2024-05-29T00:00:00.000Z	30071	Southern Star Central Gas Pipeline, Inc.	Clearwater	4	-1	Sedgwick (KS)	KS	midwest	KS	LDC	ID3	2024-05-30T21:04:59.152Z
2024-05-30T00:00:00.000Z	30073	Southern Star Central Gas Pipeline, Inc.	Cleveland Mo	0	-1	Cass (MO)				LDC	ID3	2024-05-30T21:04:59.152Z
2024-05-28T00:00:00.000Z	26553	Southern Star Central Gas Pipeline, Inc.	Transwestern Canadian - Del.	0	-1	HEMPHILL (TX)	TX	gulf	North - TX	Interconnect	ID3	2024-05-30T21:04:59.152Z
2024-05-30T00:00:00.000Z	2527	Southern Star Central Gas Pipeline, Inc.	Cowley Gas Systems, LLC	0	1	COWLEY (KS)	KS	midwest	KS	Production	ID3	2024-05-30T21:04:59.152Z

/wells

Completions