Services

CoreService

rpc GetFeastCoreVersion(GetFeastCoreVersionRequest) returns (GetFeastCoreVersionResponse)

Retrieve version information about this Feast deployment

rpc GetFeatureSet(GetFeatureSetRequest) returns (GetFeatureSetResponse)

Returns a specific feature set

rpc GetEntity(GetEntityRequest) returns (GetEntityResponse)

Returns a specific entity

rpc ListFeatureSets(ListFeatureSetsRequest) returns (ListFeatureSetsResponse)

Retrieve feature set details given a filter.

Returns all feature sets matching that filter. If none are found, an empty list will be returned. If no filter is provided in the request, the response will contain all the feature sets currently stored in the registry.

rpc ListFeatures(ListFeaturesRequest) returns (ListFeaturesResponse)

Returns all feature references and respective features matching that filter. If none are found an empty map will be returned If no filter is provided in the request, the response will contain all the features currently stored in the default project.

rpc GetFeatureStatistics(GetFeatureStatisticsRequest) returns (GetFeatureStatisticsResponse)

Get feature statistics computed over the data in the batch stores.

Returns a dataset containing TFDV statistics mapped to each valid historical store.

rpc ListStores(ListStoresRequest) returns (ListStoresResponse)

Retrieve store details given a filter.

Returns all stores matching that filter. If none are found, an empty list will be returned. If no filter is provided in the request, the response will contain all the stores currently stored in the registry.

rpc ApplyFeatureSet(ApplyFeatureSetRequest) returns (ApplyFeatureSetResponse)

Create or update and existing feature set.

This function is idempotent - it will not create a new feature set if schema does not change. Schema changes will update the feature set if the changes are valid. All changes except the following are valid: - Changes to feature set id (name, project) - Changes to entities - Changes to feature name and type

rpc ApplyEntity(ApplyEntityRequest) returns (ApplyEntityResponse)

Create or update and existing entity.

This function is idempotent - it will not create a new entity if schema does not change. Schema changes will update the entity if the changes are valid. Following changes are not valid: - Changes to name - Changes to type

rpc ListEntities(ListEntitiesRequest) returns (ListEntitiesResponse)

Returns all entity references and respective entities matching that filter. If none are found an empty map will be returned If no filter is provided in the request, the response will contain all the entities currently stored in the default project.

rpc UpdateStore(UpdateStoreRequest) returns (UpdateStoreResponse)

Updates core with the configuration of the store.

If the changes are valid, core will return the given store configuration in response, and start or update the necessary feature population jobs for the updated store.

rpc CreateProject(CreateProjectRequest) returns (CreateProjectResponse)

Creates a project. Projects serve as namespaces within which resources like features will be created. Feature set names as must be unique within a project while field (Feature/Entity) names must be unique within a Feature Set. Project names themselves must be globally unique.

rpc ArchiveProject(ArchiveProjectRequest) returns (ArchiveProjectResponse)

Archives a project. Archived projects will continue to exist and function, but won’t be visible through the Core API. Any existing ingestion or serving requests will continue to function, but will result in warning messages being logged. It is not possible to unarchive a project through the Core API

rpc ListProjects(ListProjectsRequest) returns (ListProjectsResponse)

Lists all projects active projects.

rpc UpdateFeatureSetStatus(UpdateFeatureSetStatusRequest) returns (UpdateFeatureSetStatusResponse)

Internal API for Job Controller to update featureSet’s status once responsible ingestion job is running

rpc ApplyFeatureTable(ApplyFeatureTableRequest) returns (ApplyFeatureTableResponse)

Create or update an existing feature table. This function is idempotent - it will not create a new feature table if the schema does not change. Schema changes will update the feature table if the changes are valid. All changes except the following are valid: - Changes to feature table name. - Changes to entities - Changes to feature name and type

rpc ListFeatureTables(ListFeatureTablesRequest) returns (ListFeatureTablesResponse)

List feature tables that match a given filter. Returns the references of the Feature Tables matching that filter. If none are found, an empty list will be returned. If no filter is provided in the request, the response will match all the feature tables currently stored in the registry.

rpc GetFeatureTable(GetFeatureTableRequest) returns (GetFeatureTableResponse)

Returns a specific feature table

JobControllerService

rpc ListIngestionJobs(ListIngestionJobsRequest) returns (ListIngestionJobsResponse)

List Ingestion Jobs given an optional filter. Returns allow ingestions matching the given request filter. Returns all ingestion jobs if no filter is provided. Returns an empty list if no ingestion jobs match the filter.

rpc RestartIngestionJob(RestartIngestionJobRequest) returns (RestartIngestionJobResponse)

Restart an Ingestion Job. Restarts the ingestion job with the given job id. NOTE: Data might be lost during the restart for some job runners. Does not support stopping a job in a transitional (ie pending, suspending, aborting), terminal state (ie suspended or aborted) or unknown status

rpc StopIngestionJob(StopIngestionJobRequest) returns (StopIngestionJobResponse)

Stop an Ingestion Job. Stop (Aborts) the ingestion job with the given job id. Does nothing if the target job if already in a terminal state (ie suspended or aborted). Does not support stopping a job in a transitional (ie pending, suspending, aborting) or unknown status

Types

Entity

Field	Type	Description	Required
`spec`	`EntitySpecV2`	User-specified specifications of this entity.	No
`meta`	`EntityMeta`	System-populated metadata for this entity.	No

EntitySpecV2

Field	Type	Description	Required
`name`	`string`	Name of the entity.	No
`valueType`	`Enum`	Type of the entity.	No
`description`	`string`	Description of the entity.	No
`labels`	`map<string, string>`	User defined metadata	No

EntityMeta

Field	Type	Description	Required
`createdTimestamp`	`Timestamp`		No
`lastUpdatedTimestamp`	`Timestamp`		No

Source

Field	Type	Description	Required
`type`	`SourceType`	The kind of data source Feast should connect to in order to retrieve FeatureRow value	No
`kafkaSourceConfig`	`KafkaSourceConfig (oneof)`		No

KafkaSourceConfig

Field	Type	Description	Required
`bootstrapServers`	`string`	Comma separated list of Kafka bootstrap servers. Used for feature sets without a defined source host[:port]]	No
`topic`	`string`	Kafka topic to use for feature sets without user defined topics	No
`partitions`	`int32`	Number of Kafka partitions to to use for managed feature stream.	No
`replicationFactor`	`int32`	Defines the number of copies of managed feature stream Kafka.	No

FeatureSet

Field	Type	Description	Required
`spec`	`FeatureSetSpec`	User-specified specifications of this feature set.	No
`meta`	`FeatureSetMeta`	System-populated metadata for this feature set.	No

FeatureSetSpec

Field	Type	Description	Required
`project`	`string`	Name of project that this feature set belongs to.	No
`name`	`string`	Name of the feature set. Must be unique.	No
`entities`	`EntitySpec[]`	List of entities contained within this featureSet. This allows the feature to be used during joins between feature sets. If the featureSet is ingested into a store that supports keys, this value will be made a key.	No
`features`	`FeatureSpec[]`	List of features contained within this featureSet.	No
`maxAge`	`Duration`	Features in this feature set will only be retrieved if they are found after [time - max_age]. Missing or older feature values will be returned as nulls and indicated to end user	No
`source`	`Source`	Optional. Source on which feature rows can be found. If not set, source will be set to the default value configured in Feast Core.	No
`labels`	`map<string, string>`	User defined metadata	No
`version`	`int32`	Read-only self-incrementing version that increases monotonically when changes are made to a feature set	No

EntitySpec

Field	Type	Description	Required
`name`	`string`	Name of the entity.	No
`valueType`	`Enum`	Value type of the entity.	No

FeatureSpec

Field	Type	Description	Required
`name`	`string`	Name of the feature.	No
`valueType`	`Enum`	Value type of the feature.	No
`labels`	`map<string, string>`	Labels for user defined metadata on a feature	No
`presence`	`FeaturePresence (oneof)`	Constraints on the presence of this feature in the examples.	No
`groupPresence`	`FeaturePresenceWithinGroup (oneof)`	Only used in the context of a “group” context, e.g., inside a sequence.	No
`shape`	`FixedShape (oneof)`	The feature has a fixed shape corresponding to a multi-dimensional tensor.	No
`valueCount`	`ValueCount (oneof)`	The feature doesn’t have a well defined shape. All we know are limits on the minimum and maximum number of values.	No
`domain`	`string (oneof)`	Reference to a domain defined at the schema level.	No
`intDomain`	`IntDomain (oneof)`	Inline definitions of domains.	No
`floatDomain`	`FloatDomain (oneof)`		No
`stringDomain`	`StringDomain (oneof)`		No
`boolDomain`	`BoolDomain (oneof)`		No
`structDomain`	`StructDomain (oneof)`		No
`naturalLanguageDomain`	`NaturalLanguageDomain (oneof)`	Supported semantic domains.	No
`imageDomain`	`ImageDomain (oneof)`		No
`midDomain`	`MIDDomain (oneof)`		No
`urlDomain`	`URLDomain (oneof)`		No
`timeDomain`	`TimeDomain (oneof)`		No
`timeOfDayDomain`	`TimeOfDayDomain (oneof)`		No

FeatureSetMeta

Field	Type	Description	Required
`createdTimestamp`	`Timestamp`	Created timestamp of this specific feature set.	No
`status`	`FeatureSetStatus`	Status of the feature set. Used to indicate whether the feature set is ready for consumption or ingestion. Currently supports 2 states: 1) STATUSPENDING - A feature set is in pending state if Feast has not spun up the jobs necessary to push rows for this feature set to stores subscribing to this feature set. 2) STATUSREADY - Feature set is ready for consumption or ingestion	No

tensorflow.metadata.v0.FeaturePresence

Describes constraints on the presence of the feature in the data.

Field	Type	Description	Required
`minFraction`	`double`	Minimum fraction of examples that have this feature.	No
`minCount`	`int64`	Minimum number of examples that have this feature.	No

tensorflow.metadata.v0.FeaturePresenceWithinGroup

Records constraints on the presence of a feature inside a “group” context (e.g., .presence inside a group of features that define a sequence).

Field	Type	Description	Required
`required`	`bool`		No

tensorflow.metadata.v0.FixedShape

Specifies a fixed shape for the feature’s values. The immediate implication is that each feature has a fixed number of values. Moreover, these values can be parsed in a multi-dimensional tensor using the specified axis sizes. The FixedShape defines a lexicographical ordering of the data. For instance, if there is a FixedShape { dim {size:3} dim {size:2} } then tensor0=field[0] then tensor0=field[1] then tensor1=field[2] then tensor1=field[3] then tensor2=field[4] then tensor2=field[5]

The FixedShape message is identical with the TensorFlow TensorShape proto message.

Field	Type	Description	Required
`dim`	`Dim[]`	The dimensions that define the shape. The total number of values in each example is the product of sizes of each dimension.	No

tensorflow.metadata.v0.ValueCount

Limits on maximum and minimum number of values in a single example (when the feature is present). Use this when the minimum value count can be different than the maximum value count. Otherwise prefer FixedShape.

Field	Type	Description	Required
`min`	`int64`		No
`max`	`int64`		No

tensorflow.metadata.v0.IntDomain

Encodes information for domains of integer values. Note that FeatureType could be either INT or BYTES.

Field	Type	Description	Required
`name`	`string`	Id of the domain. Required if the domain is defined at the schema level. If so, then the name must be unique within the schema.	No
`min`	`int64`	Min and max values for the domain.	No
`max`	`int64`		No
`isCategorical`	`bool`	If true then the domain encodes categorical values (i.e., ids) rather than ordinal values.	No

tensorflow.metadata.v0.FloatDomain

Encodes information for domains of float values. Note that FeatureType could be either INT or BYTES.

Field	Type	Description	Required
`name`	`string`	Id of the domain. Required if the domain is defined at the schema level. If so, then the name must be unique within the schema.	No
`min`	`float`	Min and max values of the domain.	No
`max`	`float`		No

tensorflow.metadata.v0.StringDomain

Encodes information for domains of string values.

Field	Type	Description	Required
`name`	`string`	Id of the domain. Required if the domain is defined at the schema level. If so, then the name must be unique within the schema.	No
`value`	`string[]`	The values appearing in the domain.	No

tensorflow.metadata.v0.BoolDomain

Encodes information about the domain of a boolean attribute that encodes its TRUE/FALSE values as strings, or 0=false, 1=true. Note that FeatureType could be either INT or BYTES.

Field	Type	Description	Required
`name`	`string`	Id of the domain. Required if the domain is defined at the schema level. If so, then the name must be unique within the schema.	No
`trueValue`	`string`	Strings values for TRUE/FALSE.	No
`falseValue`	`string`		No

tensorflow.metadata.v0.StructDomain

Domain for a recursive struct. NOTE: If a feature with a StructDomain is deprecated, then all the child features (features and sparse_features of the StructDomain) are also considered to be deprecated. Similarly child features can only be in environments of the parent feature.

Field	Type	Description	Required
`feature`	`Feature[]`		No
`sparseFeature`	`SparseFeature[]`		No

tensorflow.metadata.v0.NaturalLanguageDomain

Natural language text.

tensorflow.metadata.v0.ImageDomain

Image data.

tensorflow.metadata.v0.MIDDomain

Knowledge graph ID, see: https://www.wikidata.org/wiki/Property:P646

tensorflow.metadata.v0.URLDomain

A URL, see: https://en.wikipedia.org/wiki/URL

tensorflow.metadata.v0.TimeDomain

Time or date representation.

Field	Type	Description	Required
`stringFormat`	`string (oneof)`	Expected format that contains a combination of regular characters and special format specifiers. Format specifiers are a subset of the strptime standard.	No
`integerFormat`	`IntegerTimeFormat (oneof)`	Expected format of integer times.	No

tensorflow.metadata.v0.TimeOfDayDomain

Time of day, without a particular date.

Field	Type	Description	Required
`stringFormat`	`string (oneof)`	Expected format that contains a combination of regular characters and special format specifiers. Format specifiers are a subset of the strptime standard.	No
`integerFormat`	`IntegerTimeOfDayFormat (oneof)`	Expected format of integer times.	No

DataSource

Defines a Data Source that can be used source Feature data

Field	Type	Description	Required
`type`	`SourceType`		No
`fieldMapping`	`map<string, string>`	Defines mapping between fields in the sourced data and fields in parent FeatureTable.	No
`timestampColumn`	`string`	Must specify timestamp column name	No
`datePartitionColumn`	`string`	(Optional) Specify partition column useful for file sources	No
`fileOptions`	`FileOptions (oneof)`		No
`bigqueryOptions`	`BigQueryOptions (oneof)`		No
`kafkaOptions`	`KafkaOptions (oneof)`		No
`kinesisOptions`	`KinesisOptions (oneof)`		No

DataSource.FileOptions

Defines options for DataSource that sources features from a file

Field	Type	Description	Required
`fileFormat`	`string`	File Format of the file containing the features	No
`fileUrl`	`string`	Target URL of file to retrieve and source features from. s3://path/to/file for AWS S3 storage gs://path/to/file for GCP GCS storage file:///path/to/file for local storage	No

DataSource.BigQueryOptions

Defines options for DataSource that sources features from a BigQuery Query

Field	Type	Description	Required
`tableRef`	`string`	Full table reference in the form of [project:dataset.table]	No

DataSource.KafkaOptions

Defines options for DataSource that sources features from Kafka messages. Each message should be a Protobuf that can be decoded with the generated Java Protobuf class at the given class path

Field	Type	Description	Required
`bootstrapServers`	`string`	Comma separated list of Kafka bootstrap servers. Used for feature tables without a defined source host[:port]]	No
`topic`	`string`	Kafka topic to collect feature data from.	No
`classPath`	`string`	Classpath to the generated Java Protobuf class that can be used to decode Feature data from the obtained Kafka message	No

DataSource.KinesisOptions

Defines options for DataSource that sources features from Kinesis records. Each record should be a Protobuf that can be decoded with the generated Java Protobuf class at the given class path

Field	Type	Description	Required
`region`	`string`	AWS region of the Kinesis stream	No
`streamName`	`string`	Name of the Kinesis stream to obtain feature data from.	No
`classPath`	`string`	Classpath to the generated Java Protobuf class that can be used to decode Feature data from the obtained Kinesis record	No

FeatureSpecV2

Field	Type	Description	Required
`name`	`string`	Name of the feature. Not updatable.	No
`valueType`	`Enum`	Value type of the feature. Not updatable.	No
`labels`	`map<string, string>`	Labels for user defined metadata on a feature	No

FeatureTable

Field	Type	Description	Required
`spec`	`FeatureTableSpec`	User-specified specifications of this feature table.	No
`meta`	`FeatureTableMeta`	System-populated metadata for this feature table.	No

FeatureTableSpec

Field	Type	Description	Required
`name`	`string`	Name of the feature set. Must be unique. Not updated.	No
`entities`	`string[]`	List names of entities to associate with the Features defined in this Feature Table. Not updatable.	No
`features`	`FeatureSpecV2[]`	List of features specifications for each feature defined with this feature table.	No
`labels`	`map<string, string>`	User defined metadata	No
`maxAge`	`Duration`	Features in this feature table can only be retrieved from online serving younger than max age. Age is measured as the duration of time between the feature’s event timestamp and when the feature is retrieved Feature values outside max age will be returned as unset values and indicated to end user	No
`batchSource`	`DataSource`	Batch/Offline DataSource to source batch/offline feature data. Only batch DataSource can be specified (ie source type should start with ‘BATCH_’)	No
`streamSource`	`DataSource`	Stream/Online DataSource to source stream/online feature data. Only stream DataSource can be specified (ie source type should start with ‘STREAM_’)	No

FeatureTableMeta

Field	Type	Description	Required
`createdTimestamp`	`Timestamp`	Time where this Feature Table is created	No
`lastUpdatedTimestamp`	`Timestamp`	Time where this Feature Table is last updated	No
`revision`	`int64`	Auto incrementing revision no. of this Feature Table	No

Store

Store provides a location where Feast reads and writes feature values. Feature values will be written to the Store in the form of FeatureRow elements. The way FeatureRow is encoded and decoded when it is written to and read from the Store depends on the type of the Store.

For example, a FeatureRow will materialize as a row in a table in BigQuery but it will materialize as a key, value pair element in Redis.

Field	Type	Description	Required
`name`	`string`	Name of the store.	No
`type`	`StoreType`	Type of store.	No
`subscriptions`	`Subscription[]`	Feature sets to subscribe to.	No
`redisConfig`	`RedisConfig (oneof)`		No
`bigqueryConfig`	`BigQueryConfig (oneof)`		No
`cassandraConfig`	`CassandraConfig (oneof)`		No
`redisClusterConfig`	`RedisClusterConfig (oneof)`		No

Store.RedisConfig

Field	Type	Description	Required
`host`	`string`		No
`port`	`int32`		No
`initialBackoffMs`	`int32`	Optional. The number of milliseconds to wait before retrying failed Redis connection. By default, Feast uses exponential backoff policy and “initialbackoffms” sets the initial wait duration.	No
`maxRetries`	`int32`	Optional. Maximum total number of retries for connecting to Redis. Default to zero retries.	No
`flushFrequencySeconds`	`int32`	Optional. How often flush data to redis	No

Store.BigQueryConfig

Field	Type	Description	Required
`projectId`	`string`		No
`datasetId`	`string`		No
`stagingLocation`	`string`		No
`initialRetryDelaySeconds`	`int32`		No
`totalTimeoutSeconds`	`int32`		No
`writeTriggeringFrequencySeconds`	`int32`	Required. Frequency of running BQ load job and flushing all collected rows to BQ table	No

Store.CassandraConfig

Field	Type	Description	Required
`host`	`string`		No
`port`	`int32`		No

Store.RedisClusterConfig

Field	Type	Description	Required
`connectionString`	`string`	List of Redis Uri for all the nodes in Redis Cluster, comma separated. Eg. host1:6379, host2:6379	No
`initialBackoffMs`	`int32`		No
`maxRetries`	`int32`		No
`flushFrequencySeconds`	`int32`	Optional. How often flush data to redis	No
`keyPrefix`	`string`	Optional. Append a prefix to the Redis Key	No
`enableFallback`	`bool`	Optional. Enable fallback to another key prefix if the original key is not present. Useful for migrating key prefix without re-ingestion. Disabled by default.	No
`fallbackPrefix`	`string`	Optional. This would be the fallback prefix to use if enable_fallback is true.	No

Store.Subscription

Field	Type	Description	Required
`project`	`string`	Name of project that the feature sets belongs to. This can be one of - [project_name] - * If an asterisk is provided, filtering on projects will be disabled. All projects will be matched. It is NOT possible to provide an asterisk with a string in order to do pattern matching.	No
`name`	`string`	Name of the desired feature set. Asterisks can be used as wildcards in the name. Matching on names is only permitted if a specific project is defined. It is disallowed If the project name is set to “” e.g. - can be used to match all feature sets - my-feature-set* can be used to match all features prefixed by “my-feature-set” - my-feature-set-6 can be used to select a single feature set	No
`exclude`	`bool`	All matches with exclude enabled will be filtered out instead of added	No

FeatureSetReference

Defines a composite key that refers to a unique FeatureSet

Field	Type	Description	Required
`project`	`string`	Name of the project	No
`name`	`string`	Name of the FeatureSet	No

IngestionJob

Represents Feast Injestion Job

Field	Type	Description	Required
`id`	`string`	Job ID assigned by Feast	No
`externalId`	`string`	External job ID specific to the runner. For DirectRunner jobs, this is identical to id. For DataflowRunner jobs, this refers to the Dataflow job ID.	No
`status`	`IngestionJobStatus`		No
`source`	`Source`	Source this job is reading from.	No
`stores`	`Store[]`	Store this job is writing to.	No
`featureSetReferences`	`FeatureSetReference[]`	List of Feature Set References	No

SpecsStreamingUpdateConfig

Config for bi-directional communication channel between Core Service and Ingestion Job

Field	Type	Description	Required
`source`	`KafkaSourceConfig`	out-channel for publishing new FeatureSetSpecs (by Core). IngestionJob use it as source of existing FeatureSetSpecs and new real-time updates	No
`ack`	`KafkaSourceConfig`	ack-channel for sending acknowledgments when new FeatureSetSpecs is installed in Job	No

FeatureSetSpecAck

Field	Type	Required
`featureSetReference`	`string`	No
`featureSetVersion`	`int32`	No
`jobName`	`string`	No

GetFeatureSetRequest

Request for a single feature set

Field	Type	Description	Required
`project`	`string`	Name of project the feature set belongs to. If omitted will default to ‘default’ project.	No
`name`	`string`	Name of feature set (required).	No

GetFeatureSetResponse

Response containing a single feature set

Field	Type	Description	Required
`featureSet`	`FeatureSet`		No

ListFeatureSetsRequest

Retrieves details for all versions of a specific feature set

Field	Type	Description	Required
`filter`	`Filter`		No

ListFeatureSetsResponse

Field	Type	Description	Required
`featureSets`	`FeatureSet[]`		No

GetEntityRequest

Request for a single entity

Field	Type	Description	Required
`name`	`string`	Name of entity (required).	No
`project`	`string`	Name of project the entity belongs to. If omitted will default to ‘default’ project.	No

GetEntityResponse

Response containing a single entity

Field	Type	Description	Required
`entity`	`Entity`		No

ListEntitiesRequest

Retrieves details for all versions of a specific entity

Field	Type	Description	Required
`filter`	`Filter`		No

ListEntitiesResponse

Field	Type	Description	Required
`entities`	`Entity[]`		No

ListFeaturesRequest

Field	Type	Description	Required
`filter`	`Filter`		No

ListFeaturesResponse

Field	Type	Description	Required
`features`	`map<string, FeatureSpec>`		No

ListStoresRequest

Field	Type	Description	Required
`filter`	`Filter`		No

ListStoresResponse

Field	Type	Description	Required
`store`	`Store[]`		No

ApplyEntityRequest

Field	Type	Description	Required
`spec`	`EntitySpecV2`	If project is unspecified, will default to ‘default’ project. If project specified does not exist, the project would be automatically created.	No
`project`	`string`	Name of project that this entity belongs to.	No

ApplyEntityResponse

Field	Type	Description	Required
`entity`	`Entity`		No

ApplyFeatureSetRequest

Field	Type	Description	Required
`featureSet`	`FeatureSet`	Feature set version If project is unspecified, will default to ‘default’ project. If project specified does not exist, the project would be automatically created.	No

ApplyFeatureSetResponse

Field	Type	Description	Required
`featureSet`	`FeatureSet`		No
`status`	`Status`		No

GetFeastCoreVersionRequest

GetFeastCoreVersionResponse

Field	Type	Description	Required
`version`	`string`		No

UpdateStoreRequest

Field	Type	Description	Required
`store`	`Store`		No

UpdateStoreResponse

Field	Type	Description	Required
`store`	`Store`		No
`status`	`Status`		No

CreateProjectRequest

Request to create a project

Field	Type	Description	Required
`name`	`string`	Name of project (required)	No

CreateProjectResponse

Response for creation of a project

ArchiveProjectRequest

Request for the archival of a project

Field	Type	Description	Required
`name`	`string`	Name of project to be archived	No

ArchiveProjectResponse

Response for archival of a project

ListProjectsRequest

Request for listing of projects

ListProjectsResponse

Response for listing of projects

Field	Type	Description	Required
`projects`	`string[]`	List of project names (archived projects are filtered out)	No

ListIngestionJobsRequest

Request for listing ingestion jobs

Field	Type	Description	Required
`filter`	`Filter`		No

ListIngestionJobsResponse

Response from listing ingestion jobs

Field	Type	Description	Required
`jobs`	`IngestionJob[]`		No

RestartIngestionJobRequest

Request to restart ingestion job

Field	Type	Description	Required
`id`	`string`	Job ID assigned by Feast	No

RestartIngestionJobResponse

Response from restartingan injestion job

StopIngestionJobRequest

Request to stop ingestion job

Field	Type	Description	Required
`id`	`string`	Job ID assigned by Feast	No

StopIngestionJobResponse

Request from stopping an ingestion job

GetFeatureStatisticsRequest

Field	Type	Description	Required
`featureSetId`	`string`	Feature set to retrieve the statistics for. A fully qualified feature set id in the format of project/feature_set must be provided.	No
`features`	`string[]`	Optional filter which filters returned statistics by selected features. These features must be present in the data that is being processed.	No
`store`	`string`	Optional filter to select store over which the statistics will retrieved. Only historical stores are allowed.	No
`startDate`	`Timestamp`	Optional start and end dates over which to filter statistical data Start date is inclusive, but end date is not. Only dates are supported, not times. Cannot be used with dataset_ids. If this period spans multiple days, unaggregatable statistics will be dropped.	No
`endDate`	`Timestamp`		No
`ingestionIds`	`string[]`	Optional list of ingestion Ids by which to filter data before retrieving statistics. Cannot be used with the date ranges If multiple dataset ids are provided, unaggregatable statistics will be dropped.	No
`forceRefresh`	`bool`	Setting this flag to true will force a recalculation of statistics and overwrite results currently in the cache, if any.	No

GetFeatureStatisticsResponse

Field	Type	Description	Required
`datasetFeatureStatisticsList`	`DatasetFeatureStatisticsList`	Contains statistics for the requested data. Due to the limitations of TFDV and Facets, only a single dataset can be returned in, despite the message being of list type.	No

UpdateFeatureSetStatusRequest

Field	Type	Description	Required
`reference`	`FeatureSetReference`	FeatureSetReference of FeatureSet to update	No
`status`	`FeatureSetStatus`	Target status	No

UpdateFeatureSetStatusResponse

ApplyFeatureTableRequest

Field	Type	Description	Required
`project`	`string`	Optional. Name of the Project to apply the Feature Table to. If unspecified, will apply FeatureTable to the default project.	No
`tableSpec`	`FeatureTableSpec`	Feature Table specification to apply	No

ApplyFeatureTableResponse

Field	Type	Description	Required
`table`	`FeatureTable`		No

GetFeatureTableRequest

Field	Type	Description	Required
`project`	`string`	Optional. Name of the Project to retrieve the Feature Table from. If unspecified, will apply FeatureTable to the default project.	No
`name`	`string`	Name of the FeatureTable to retrieve.	No

GetFeatureTableResponse

Field	Type	Description	Required
`table`	`FeatureTable`	The Feature Table retrieved.	No

ListFeatureTablesRequest

Field	Type	Description	Required
`filter`	`Filter`	Filter used when listing Feature Tables	No

ListFeatureTablesResponse

Field	Type	Description	Required
`tables`	`FeatureTable[]`	List of matching Feature Tables	No

ListFeatureSetsRequest.Filter

Field	Type	Description	Required
`project`	`string`	Name of project that the feature sets belongs to. This can be one of - [project_name] - * If an asterisk is provided, filtering on projects will be disabled. All projects will be matched. It is NOT possible to provide an asterisk with a string in order to do pattern matching. If unspecified this field will default to the default project ‘default’.	No
`featureSetName`	`string`	Name of the desired feature set. Asterisks can be used as wildcards in the name. Matching on names is only permitted if a specific project is defined. It is disallowed If the project name is set to “” e.g. - can be used to match all feature sets - my-feature-set* can be used to match all features prefixed by “my-feature-set” - my-feature-set-6 can be used to select a single feature set	No
`labels`	`map<string, string>`	User defined metadata for feature set. Feature sets with all matching labels will be returned.	No
`status`	`FeatureSetStatus`	Filter by FeatureSet’s current status Project and Feature Set name still must be specified (could be “*”)	No

ListEntitiesRequest.Filter

Field	Type	Description	Required
`project`	`string`	Optional. Specifies the name of the project to list Entities in. It is NOT possible to provide an asterisk with a string in order to do pattern matching. If unspecified, this field will default to the default project ‘default’.	No
`labels`	`map<string, string>`	Optional. User defined metadata for entity. Entities with all matching labels will be returned.	No

ListFeaturesRequest.Filter

Field	Type	Description	Required
`labels`	`map<string, string>`	User defined metadata for feature. Features with all matching labels will be returned.	No
`entities`	`string[]`	List of entities contained within the featureSet that the feature belongs to. Only feature sets with these entities will be searched for features.	No
`project`	`string`	Name of project that the feature sets belongs to. Filtering on projects is disabled. It is NOT possible to provide an asterisk with a string in order to do pattern matching. If unspecified this field will default to the default project ‘default’.	No

ListStoresRequest.Filter

Field	Type	Description	Required
`name`	`string`	Name of desired store. Regex is not supported in this query.	No

ListIngestionJobsRequest.Filter

Field	Type	Description	Required
`id`	`string`	Filter by Job ID assigned by Feast	No
`featureSetReference`	`FeatureSetReference`	Filter by ingestion job target feature set.	No
`storeName`	`string`	Filter by Name of store	No

ListFeatureTablesRequest.Filter

Field	Type	Description	Required
`project`	`string`	Optional. Specifies the name of the project to list Feature Tables in. If unspecified would list Feature Tables in the default project.	No
`labels`	`map<string, string>`	Optional. Feature Tables with all matching labels will be returned. If unspecified would list Feature Tables without filtering by labels.	No

tensorflow.metadata.v0.DatasetFeatureStatisticsList

A list of features statistics for different datasets. If you wish to compare different datasets using this list, then the DatasetFeatureStatistics entries should all contain the same list of features.

Field	Type	Description	Required
`datasets`	`DatasetFeatureStatistics[]`		No

DirectRunnerConfigOptions

Field	Type	Description	Required
`targetParallelism`	`int32`	* Controls the amount of target parallelism the DirectRunner will use. Defaults to the greater of the number of available processors and 3. Must be a value greater than zero.	No
`deadLetterTableSpec`	`string`	BigQuery table specification, e.g. PROJECTID:DATASETID.PROJECT_ID	No
`tempLocation`	`string`	A pipeline level default location for storing temporary files. Support Google Cloud Storage locations or local path	No

DataflowRunnerConfigOptions

Field	Type	Description	Required
`project`	`string`	Project id to use when launching jobs.	No
`region`	`string`	The Google Compute Engine region for creating Dataflow jobs.	No
`workerZone`	`string`	GCP availability zone for operations.	No
`serviceAccount`	`string`	Run the job as a specific service account, instead of the default GCE robot.	No
`network`	`string`	GCE network for launching workers.	No
`subnetwork`	`string`	GCE subnetwork for launching workers. e.g. regions/asia-east1/subnetworks/mysubnetwork	No
`workerMachineType`	`string`	Machine type to create Dataflow worker VMs as.	No
`autoscalingAlgorithm`	`string`	The autoscaling algorithm to use for the workerpool.	No
`usePublicIps`	`bool`	Specifies whether worker pools should be started with public IP addresses.	No
`tempLocation`	`string`	A pipeline level default location for storing temporary files. Support Google Cloud Storage locations, e.g. gs://bucket/object	No
`maxNumWorkers`	`int32`	The maximum number of workers to use for the workerpool.	No
`deadLetterTableSpec`	`string`	BigQuery table specification, e.g. PROJECTID:DATASETID.PROJECT_ID	No
`labels`	`map<string, string>`	Labels to apply to the dataflow job	No
`diskSizeGb`	`int32`	Disk size to use on each remote Compute Engine worker instance	No
`enableStreamingEngine`	`bool`	Run job on Dataflow Streaming Engine instead of creating worker VMs	No
`workerDiskType`	`string`	Type of persistent disk to be used by workers	No
`kafkaConsumerProperties`	`map<string, string>`	Kafka consumer configuration properties	No

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

SourceType

Name	Description
`INVALID`
`KAFKA`

SourceType

Name	Description
`INVALID`
`KAFKA`

FeatureSetStatus

Name	Description
`STATUS_INVALID`
`STATUS_PENDING`
`STATUS_JOB_STARTING`
`STATUS_READY`

FeatureSetJobDeliveryStatus

Name	Description
`STATUS_IN_PROGRESS`
`STATUS_DELIVERED`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

FeatureSetStatus

Name	Description
`STATUS_INVALID`
`STATUS_PENDING`
`STATUS_JOB_STARTING`
`STATUS_READY`

DataSource.SourceType

Type of Data Source.

Name	Description
`INVALID`
`BATCH_FILE`
`BATCH_BIGQUERY`
`STREAM_KAFKA`
`STREAM_KINESIS`

DataSource.SourceType

Type of Data Source.

Name	Description
`INVALID`
`BATCH_FILE`
`BATCH_BIGQUERY`
`STREAM_KAFKA`
`STREAM_KINESIS`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

Store.StoreType

Name	Description
`INVALID`
`REDIS`	Redis stores a FeatureRow element as a key, value pair. The Redis data types used (https://redis.io/topics/data-types): - key: STRING - value: STRING Encodings: - key: byte array of RedisKey (refer to feast.storage.RedisKey) - value: byte array of FeatureRow (refer to feast.types.FeatureRow)
`BIGQUERY`	BigQuery stores a FeatureRow element as a row in a BigQuery table. Table name is derived is the same as the feature set name. The entities and features in a FeatureSetSpec corresponds to the fields in the BigQuery table (these make up the BigQuery schema). The name of the entity spec and feature spec corresponds to the column names, and the value_type of entity spec and feature spec corresponds to BigQuery standard SQL data type of the column. The following BigQuery fields are reserved for Feast internal use. Ingestion of entity or feature spec with names identical to the following field names will raise an exception during ingestion. columnname \| columndatatype \| description ====================\|==================\|================================ - eventtimestamp \| TIMESTAMP \| event time of the FeatureRow - createdtimestamp \| TIMESTAMP \| processing time of the ingestion of the FeatureRow - ingestionid \| STRING \| unique id identifying groups of rows that have been ingested together - job_id \| STRING \| identifier for the job that writes the FeatureRow to the corresponding BigQuery table BigQuery table created will be partitioned by the field “event_timestamp” of the FeatureRow (https://cloud.google.com/bigquery/docs/partitioned-tables). The following table shows how ValueType in Feast is mapped to BigQuery Standard SQL data types (https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types): BYTES : BYTES STRING : STRING INT32 : INT64 INT64 : IN64 DOUBLE : FLOAT64 FLOAT : FLOAT64 BOOL : BOOL BYTESLIST : ARRAY STRINGLIST : ARRAY INT32LIST : ARRAY INT64LIST : ARRAY DOUBLELIST : ARRAY FLOATLIST : ARRAY BOOL_LIST : ARRAY The column mode in BigQuery is set to “Nullable” such that unset Value in a FeatureRow corresponds to NULL value in BigQuery.
`CASSANDRA`	Unsupported in Feast 0.3
`REDIS_CLUSTER`

Store.StoreType

Name	Description
`INVALID`
`REDIS`	Redis stores a FeatureRow element as a key, value pair. The Redis data types used (https://redis.io/topics/data-types): - key: STRING - value: STRING Encodings: - key: byte array of RedisKey (refer to feast.storage.RedisKey) - value: byte array of FeatureRow (refer to feast.types.FeatureRow)
`BIGQUERY`	BigQuery stores a FeatureRow element as a row in a BigQuery table. Table name is derived is the same as the feature set name. The entities and features in a FeatureSetSpec corresponds to the fields in the BigQuery table (these make up the BigQuery schema). The name of the entity spec and feature spec corresponds to the column names, and the value_type of entity spec and feature spec corresponds to BigQuery standard SQL data type of the column. The following BigQuery fields are reserved for Feast internal use. Ingestion of entity or feature spec with names identical to the following field names will raise an exception during ingestion. columnname \| columndatatype \| description ====================\|==================\|================================ - eventtimestamp \| TIMESTAMP \| event time of the FeatureRow - createdtimestamp \| TIMESTAMP \| processing time of the ingestion of the FeatureRow - ingestionid \| STRING \| unique id identifying groups of rows that have been ingested together - job_id \| STRING \| identifier for the job that writes the FeatureRow to the corresponding BigQuery table BigQuery table created will be partitioned by the field “event_timestamp” of the FeatureRow (https://cloud.google.com/bigquery/docs/partitioned-tables). The following table shows how ValueType in Feast is mapped to BigQuery Standard SQL data types (https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types): BYTES : BYTES STRING : STRING INT32 : INT64 INT64 : IN64 DOUBLE : FLOAT64 FLOAT : FLOAT64 BOOL : BOOL BYTESLIST : ARRAY STRINGLIST : ARRAY INT32LIST : ARRAY INT64LIST : ARRAY DOUBLELIST : ARRAY FLOATLIST : ARRAY BOOL_LIST : ARRAY The column mode in BigQuery is set to “Nullable” such that unset Value in a FeatureRow corresponds to NULL value in BigQuery.
`CASSANDRA`	Unsupported in Feast 0.3
`REDIS_CLUSTER`

IngestionJobStatus

Status of a Feast Ingestion Job

Name	Description
`UNKNOWN`	Job status is not known.
`PENDING`	Import job is submitted to runner and currently pending for executing
`RUNNING`	Import job is currently running in the runner
`COMPLETED`	Runner’s reported the import job has completed (applicable to batch job)
`ABORTING`	When user sent abort command, but it’s still running
`ABORTED`	User initiated abort job
`ERROR`	Runner’s reported that the import job failed to run or there is a failure during job
`SUSPENDING`	job has been suspended and waiting for cleanup
`SUSPENDED`	job has been suspended

IngestionJobStatus

Status of a Feast Ingestion Job

Name	Description
`UNKNOWN`	Job status is not known.
`PENDING`	Import job is submitted to runner and currently pending for executing
`RUNNING`	Import job is currently running in the runner
`COMPLETED`	Runner’s reported the import job has completed (applicable to batch job)
`ABORTING`	When user sent abort command, but it’s still running
`ABORTED`	User initiated abort job
`ERROR`	Runner’s reported that the import job failed to run or there is a failure during job
`SUSPENDING`	job has been suspended and waiting for cleanup
`SUSPENDED`	job has been suspended

ApplyFeatureSetResponse.Status

TODO: 0 should correspond to invalid rather than NO_CHANGE

Name	Description
`NO_CHANGE`	Latest feature set is consistent with provided feature set
`CREATED`	New feature set created
`ERROR`	Error occurred while trying to apply changes
`UPDATED`	Changes detected and updated successfully

UpdateStoreResponse.Status

Name	Description
`NO_CHANGE`	Existing store config matching the given store id is identical to the given store config.
`UPDATED`	New store created or existing config updated.

ApplyFeatureSetResponse.Status

TODO: 0 should correspond to invalid rather than NO_CHANGE

Name	Description
`NO_CHANGE`	Latest feature set is consistent with provided feature set
`CREATED`	New feature set created
`ERROR`	Error occurred while trying to apply changes
`UPDATED`	Changes detected and updated successfully

UpdateStoreResponse.Status

Name	Description
`NO_CHANGE`	Existing store config matching the given store id is identical to the given store config.
`UPDATED`	New store created or existing config updated.

FeatureSetStatus

Name	Description
`STATUS_INVALID`
`STATUS_PENDING`
`STATUS_JOB_STARTING`
`STATUS_READY`

FeatureSetStatus

Name	Description
`STATUS_INVALID`
`STATUS_PENDING`
`STATUS_JOB_STARTING`
`STATUS_READY`