Skip to content

Commit c8850bb

Browse files
[SDP] GraphElement
1 parent 477c7d3 commit c8850bb

File tree

6 files changed

+39
-6
lines changed

6 files changed

+39
-6
lines changed

docs/declarative-pipelines/FlowAnalysis.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ In the end, `FlowFunction` creates a [FlowFunctionResult](FlowFunctionResult.md)
3131
`createFlowFunctionFromLogicalPlan` is used when:
3232

3333
* `PipelinesHandler` is requested to [define a flow](PipelinesHandler.md#defineFlow)
34-
* `SqlGraphRegistrationContext` is requested to [handle the following logical commands](SqlGraphRegistrationContext.md#processSqlQuery):
34+
* `SqlGraphRegistrationContext` is requested to [handle the following queries](SqlGraphRegistrationContext.md#processSqlQuery):
3535
* [CreateFlowCommand](SqlGraphRegistrationContext.md#CreateFlowCommand)
3636
* [CreateMaterializedViewAsSelect](SqlGraphRegistrationContext.md#CreateMaterializedViewAsSelect)
3737
* [CreateView](SqlGraphRegistrationContext.md#CreateView)
Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,33 @@
11
# GraphElement
22

3-
`GraphElement` is...FIXME
3+
`GraphElement` is an [abstraction](#contract) of [dataflow graph elements](#implementations) that have an [identifier](#identifier) and an [origin](#origin).
4+
5+
## Contract
6+
7+
### Identifier { #identifier }
8+
9+
```scala
10+
identifier: TableIdentifier
11+
```
12+
13+
### QueryOrigin { #origin }
14+
15+
```scala
16+
origin: QueryOrigin
17+
```
18+
19+
Used when:
20+
21+
* `PipelinesHandler` is requested to [define a flow](PipelinesHandler.md#defineFlow) and an [output](PipelinesHandler.md#defineOutput)
22+
* `SqlGraphRegistrationContext` is requested to [process a single SQL query](SqlGraphRegistrationContext.md#processSqlQuery)
23+
* `FlowResolver` is requested to [attemptResolveFlow](FlowResolver.md#attemptResolveFlow)
24+
* `DatasetManager` is requested to [materialize datasets](DatasetManager.md#materializeDatasets) (for error reporting)
25+
* `FlowExecution` is requested for the [QueryOrigin](FlowExecution.md#getOrigin)
26+
* [FlowProgressEventLogger](FlowProgressEventLogger.md) is requested to report an event
27+
28+
## Implementations
29+
30+
* [Flow](Flow.md)
31+
* [Input](Input.md)
32+
* [Sink](Sink.md)
33+
* [View](View.md)

docs/declarative-pipelines/GraphRegistrationContext.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ registerFlow(
105105
`registerFlow` is used when:
106106

107107
* `PipelinesHandler` is requested to [define a flow](PipelinesHandler.md#defineFlow)
108-
* `SqlGraphRegistrationContext` is requested to [process the following logical commands](SqlGraphRegistrationContext.md#processSqlQuery):
108+
* `SqlGraphRegistrationContext` is requested to [process the following SQL queries](SqlGraphRegistrationContext.md#processSqlQuery):
109109
* [CREATE FLOW ... AS INSERT INTO ... BY NAME](../logical-operators/CreateFlowCommand.md)
110110
* [CREATE MATERIALIZED VIEW ... AS](../logical-operators/CreateMaterializedViewAsSelect.md)
111111
* [CREATE STREAMING TABLE ... AS](../logical-operators/CreateStreamingTableAsSelect.md)
@@ -141,7 +141,7 @@ registerTable(
141141
`registerTable` is used when:
142142

143143
* `PipelinesHandler` is requested to [define an output](PipelinesHandler.md#defineOutput)
144-
* `SqlGraphRegistrationContext` is requested to [process the following logical commands](SqlGraphRegistrationContext.md#processSqlQuery):
144+
* `SqlGraphRegistrationContext` is requested to [process the following SQL queries](SqlGraphRegistrationContext.md#processSqlQuery):
145145
* [CREATE MATERIALIZED VIEW ... AS](../logical-operators/CreateMaterializedViewAsSelect.md)
146146
* [CREATE STREAMING TABLE ... AS](../logical-operators/CreateStreamingTableAsSelect.md)
147147
* [CREATE STREAMING TABLE](../logical-operators/CreateStreamingTable.md)

docs/declarative-pipelines/SqlGraphRegistrationContext.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ processSqlFile(
3737

3838
* `PipelinesHandler` is requested to [defineSqlGraphElements](PipelinesHandler.md#defineSqlGraphElements)
3939

40-
### Process Single Logical Command { #processSqlQuery }
40+
### Process Single SQL Query { #processSqlQuery }
4141

4242
```scala
4343
processSqlQuery(

docs/declarative-pipelines/UnresolvedFlow.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
`UnresolvedFlow` is created when:
3030

3131
* `PipelinesHandler` is requested to [define a flow](PipelinesHandler.md#defineFlow)
32-
* `SqlGraphRegistrationContext` is requested to [handle the following logical commands](SqlGraphRegistrationContext.md#processSqlQuery):
32+
* `SqlGraphRegistrationContext` is requested to [handle the following SQL queries](SqlGraphRegistrationContext.md#processSqlQuery):
3333
* [CreateFlowCommand](SqlGraphRegistrationContext.md#CreateFlowCommand)
3434
* [CreateMaterializedViewAsSelect](SqlGraphRegistrationContext.md#CreateMaterializedViewAsSelect)
3535
* [CreateView](SqlGraphRegistrationContext.md#CreateView)

docs/declarative-pipelines/View.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# View
2+
3+
`View` is...FIXME

0 commit comments

Comments
 (0)