Services

CoreService

rpc GetFeastCoreVersion(GetFeastCoreVersionRequest) returns (GetFeastCoreVersionResponse)

Retrieve version information about this Feast deployment

rpc GetEntity(GetEntityRequest) returns (GetEntityResponse)

Returns a specific entity

rpc ListFeatures(ListFeaturesRequest) returns (ListFeaturesResponse)

Returns all feature references and respective features matching that filter. If none are found an empty map will be returned If no filter is provided in the request, the response will contain all the features currently stored in the default project.

rpc ListStores(ListStoresRequest) returns (ListStoresResponse)

Retrieve store details given a filter.

Returns all stores matching that filter. If none are found, an empty list will be returned. If no filter is provided in the request, the response will contain all the stores currently stored in the registry.

rpc ApplyEntity(ApplyEntityRequest) returns (ApplyEntityResponse)

Create or update and existing entity.

This function is idempotent - it will not create a new entity if schema does not change. Schema changes will update the entity if the changes are valid. Following changes are not valid: - Changes to name - Changes to type

rpc ListEntities(ListEntitiesRequest) returns (ListEntitiesResponse)

Returns all entity references and respective entities matching that filter. If none are found an empty map will be returned If no filter is provided in the request, the response will contain all the entities currently stored in the default project.

rpc UpdateStore(UpdateStoreRequest) returns (UpdateStoreResponse)

Updates core with the configuration of the store.

If the changes are valid, core will return the given store configuration in response, and start or update the necessary feature population jobs for the updated store.

rpc CreateProject(CreateProjectRequest) returns (CreateProjectResponse)

Creates a project. Projects serve as namespaces within which resources like features will be created. Feature table names as must be unique within a project while field (Feature/Entity) names must be unique within a Feature Table. Project names themselves must be globally unique.

rpc ArchiveProject(ArchiveProjectRequest) returns (ArchiveProjectResponse)

Archives a project. Archived projects will continue to exist and function, but won’t be visible through the Core API. Any existing ingestion or serving requests will continue to function, but will result in warning messages being logged. It is not possible to unarchive a project through the Core API

rpc ListProjects(ListProjectsRequest) returns (ListProjectsResponse)

Lists all projects active projects.

rpc ApplyFeatureTable(ApplyFeatureTableRequest) returns (ApplyFeatureTableResponse)

Create or update an existing feature table. This function is idempotent - it will not create a new feature table if the schema does not change. Schema changes will update the feature table if the changes are valid. All changes except the following are valid: - Changes to feature table name. - Changes to entities - Changes to feature name and type

rpc ListFeatureTables(ListFeatureTablesRequest) returns (ListFeatureTablesResponse)

List feature tables that match a given filter. Returns the references of the Feature Tables matching that filter. If none are found, an empty list will be returned. If no filter is provided in the request, the response will match all the feature tables currently stored in the registry.

rpc GetFeatureTable(GetFeatureTableRequest) returns (GetFeatureTableResponse)

Returns a specific feature table

rpc DeleteFeatureTable(DeleteFeatureTableRequest) returns (DeleteFeatureTableResponse)

Delete a specific feature table

JobService

rpc StartOfflineToOnlineIngestionJob(StartOfflineToOnlineIngestionJobRequest) returns (StartOfflineToOnlineIngestionJobResponse)

Start job to ingest data from offline store into online store

rpc GetHistoricalFeatures(GetHistoricalFeaturesRequest) returns (GetHistoricalFeaturesResponse)

Produce a training dataset, return a job id that will provide a file reference

rpc StartStreamToOnlineIngestionJob(StartStreamToOnlineIngestionJobRequest) returns (StartStreamToOnlineIngestionJobResponse)

Start job to ingest data from stream into online store

rpc ListJobs(ListJobsRequest) returns (ListJobsResponse)

List all types of jobs

rpc CancelJob(CancelJobRequest) returns (CancelJobResponse)

Cancel a single job

rpc GetJob(GetJobRequest) returns (GetJobResponse)

Get details of a single job

Types

Entity

Field	Type	Description	Required
`spec`	`EntitySpecV2`	User-specified specifications of this entity.	No
`meta`	`EntityMeta`	System-populated metadata for this entity.	No

EntitySpecV2

Field	Type	Description	Required
`name`	`string`	Name of the entity.	No
`valueType`	`Enum`	Type of the entity.	No
`description`	`string`	Description of the entity.	No
`labels`	`map<string, string>`	User defined metadata	No

EntityMeta

Field	Type	Description	Required
`createdTimestamp`	`Timestamp`		No
`lastUpdatedTimestamp`	`Timestamp`		No

FeatureSpecV2

Field	Type	Description	Required
`name`	`string`	Name of the feature. Not updatable.	No
`valueType`	`Enum`	Value type of the feature. Not updatable.	No
`labels`	`map<string, string>`	Labels for user defined metadata on a feature	No

FileFormat

Defines the file format encoding the features/entity data in files

Field	Type	Description	Required
`parquetFormat`	`ParquetFormat (oneof)`		No

StreamFormat

Defines the data format encoding features/entity data in data streams

Field	Type	Description	Required
`avroFormat`	`AvroFormat (oneof)`		No
`protoFormat`	`ProtoFormat (oneof)`		No

FileFormat.ParquetFormat

Defines options for the Parquet data format

StreamFormat.ProtoFormat

Defines options for the protobuf data format

Field	Type	Description	Required
`classPath`	`string`	Classpath to the generated Java Protobuf class that can be used to decode Feature data from the obtained stream message	No

StreamFormat.AvroFormat

Defines options for the avro data format

Field	Type	Description	Required
`schemaJson`	`string`	Optional if used in a File DataSource as schema is embedded in avro file. Specifies the schema of the Avro message as JSON string.	No

DataSource

Defines a Data Source that can be used source Feature data

Field	Type	Description	Required
`type`	`SourceType`		No
`fieldMapping`	`map<string, string>`	Defines mapping between fields in the sourced data and fields in parent FeatureTable.	No
`eventTimestampColumn`	`string`	Must specify event timestamp column name	No
`datePartitionColumn`	`string`	(Optional) Specify partition column useful for file sources	No
`createdTimestampColumn`	`string`	Must specify creation timestamp column name	No
`fileOptions`	`FileOptions (oneof)`		No
`bigqueryOptions`	`BigQueryOptions (oneof)`		No
`kafkaOptions`	`KafkaOptions (oneof)`		No
`kinesisOptions`	`KinesisOptions (oneof)`		No

DataSource.FileOptions

Defines options for DataSource that sources features from a file

Field	Type	Description	Required
`fileFormat`	`FileFormat`		No
`fileUrl`	`string`	Target URL of file to retrieve and source features from. s3://path/to/file for AWS S3 storage gs://path/to/file for GCP GCS storage file:///path/to/file for local storage	No

DataSource.BigQueryOptions

Defines options for DataSource that sources features from a BigQuery Query

Field	Type	Description	Required
`tableRef`	`string`	Full table reference in the form of [project:dataset.table]	No

DataSource.KafkaOptions

Defines options for DataSource that sources features from Kafka messages. Each message should be a Protobuf that can be decoded with the generated Java Protobuf class at the given class path

Field	Type	Description	Required
`bootstrapServers`	`string`	Comma separated list of Kafka bootstrap servers. Used for feature tables without a defined source host[:port]]	No
`topic`	`string`	Kafka topic to collect feature data from.	No
`messageFormat`	`StreamFormat`	Defines the stream data format encoding feature/entity data in Kafka messages.	No

DataSource.KinesisOptions

Defines options for DataSource that sources features from Kinesis records. Each record should be a Protobuf that can be decoded with the generated Java Protobuf class at the given class path

Field	Type	Description	Required
`region`	`string`	AWS region of the Kinesis stream	No
`streamName`	`string`	Name of the Kinesis stream to obtain feature data from.	No
`recordFormat`	`StreamFormat`	Defines the data format encoding the feature/entity data in Kinesis records. Kinesis Data Sources support Avro and Proto as data formats.	No

FeatureTable

Field	Type	Description	Required
`spec`	`FeatureTableSpec`	User-specified specifications of this feature table.	No
`meta`	`FeatureTableMeta`	System-populated metadata for this feature table.	No

FeatureTableSpec

Field	Type	Description	Required
`name`	`string`	Name of the feature table. Must be unique. Not updated.	No
`entities`	`string[]`	List names of entities to associate with the Features defined in this Feature Table. Not updatable.	No
`features`	`FeatureSpecV2[]`	List of features specifications for each feature defined with this feature table.	No
`labels`	`map<string, string>`	User defined metadata	No
`maxAge`	`Duration`	Features in this feature table can only be retrieved from online serving younger than max age. Age is measured as the duration of time between the feature’s event timestamp and when the feature is retrieved Feature values outside max age will be returned as unset values and indicated to end user	No
`batchSource`	`DataSource`	Batch/Offline DataSource to source batch/offline feature data. Only batch DataSource can be specified (ie source type should start with ‘BATCH_’)	No
`streamSource`	`DataSource`	Stream/Online DataSource to source stream/online feature data. Only stream DataSource can be specified (ie source type should start with ‘STREAM_’)	No

FeatureTableMeta

Field	Type	Description	Required
`createdTimestamp`	`Timestamp`	Time where this Feature Table is created	No
`lastUpdatedTimestamp`	`Timestamp`	Time where this Feature Table is last updated	No
`revision`	`int64`	Auto incrementing revision no. of this Feature Table	No
`hash`	`string`	Hash entities, features, batchsource and streamsource to inform JobService if jobs should be restarted should hash change	No

Store

Store provides a location where Feast reads and writes feature values. Feature values will be written to the Store in the form of FeatureRow elements. The way FeatureRow is encoded and decoded when it is written to and read from the Store depends on the type of the Store.

Field	Type	Description	Required
`name`	`string`	Name of the store.	No
`type`	`StoreType`	Type of store.	No
`subscriptions`	`Subscription[]`	Feature sets to subscribe to.	No
`redisConfig`	`RedisConfig (oneof)`		No
`redisClusterConfig`	`RedisClusterConfig (oneof)`		No

Store.RedisConfig

Field	Type	Description	Required
`host`	`string`		No
`port`	`int32`		No
`initialBackoffMs`	`int32`	Optional. The number of milliseconds to wait before retrying failed Redis connection. By default, Feast uses exponential backoff policy and “initialbackoffms” sets the initial wait duration.	No
`maxRetries`	`int32`	Optional. Maximum total number of retries for connecting to Redis. Default to zero retries.	No
`flushFrequencySeconds`	`int32`	Optional. How often flush data to redis	No
`ssl`	`bool`	Optional. Connect over SSL.	No

Store.RedisClusterConfig

Field	Type	Description	Required
`connectionString`	`string`	List of Redis Uri for all the nodes in Redis Cluster, comma separated. Eg. host1:6379, host2:6379	No
`initialBackoffMs`	`int32`		No
`maxRetries`	`int32`		No
`flushFrequencySeconds`	`int32`	Optional. How often flush data to redis	No
`keyPrefix`	`string`	Optional. Append a prefix to the Redis Key	No
`enableFallback`	`bool`	Optional. Enable fallback to another key prefix if the original key is not present. Useful for migrating key prefix without re-ingestion. Disabled by default.	No
`fallbackPrefix`	`string`	Optional. This would be the fallback prefix to use if enable_fallback is true.	No

Store.Subscription

Field	Type	Description	Required
`project`	`string`	Name of project that the feature sets belongs to. This can be one of - [project_name] - * If an asterisk is provided, filtering on projects will be disabled. All projects will be matched. It is NOT possible to provide an asterisk with a string in order to do pattern matching.	No
`name`	`string`	Name of the desired feature set. Asterisks can be used as wildcards in the name. Matching on names is only permitted if a specific project is defined. It is disallowed If the project name is set to “” e.g. - can be used to match all feature sets - my-feature-set* can be used to match all features prefixed by “my-feature-set” - my-feature-set-6 can be used to select a single feature set	No
`exclude`	`bool`	All matches with exclude enabled will be filtered out instead of added	No

GetEntityRequest

Request for a single entity

Field	Type	Description	Required
`name`	`string`	Name of entity (required).	No
`project`	`string`	Name of project the entity belongs to. If omitted will default to ‘default’ project.	No

GetEntityResponse

Response containing a single entity

Field	Type	Description	Required
`entity`	`Entity`		No

ListEntitiesRequest

Retrieves details for all versions of a specific entity

Field	Type	Description	Required
`filter`	`Filter`		No

ListEntitiesResponse

Field	Type	Description	Required
`entities`	`Entity[]`		No

ListFeaturesRequest

Field	Type	Description	Required
`filter`	`Filter`		No

ListFeaturesResponse

Field	Type	Description	Required
`features`	`map<string, FeatureSpecV2>`		No

ListStoresRequest

Field	Type	Description	Required
`filter`	`Filter`		No

ListStoresResponse

Field	Type	Description	Required
`store`	`Store[]`		No

ApplyEntityRequest

Field	Type	Description	Required
`spec`	`EntitySpecV2`	If project is unspecified, will default to ‘default’ project. If project specified does not exist, the project would be automatically created.	No
`project`	`string`	Name of project that this entity belongs to.	No

ApplyEntityResponse

Field	Type	Description	Required
`entity`	`Entity`		No

GetFeastCoreVersionRequest

GetFeastCoreVersionResponse

Field	Type	Description	Required
`version`	`string`		No

UpdateStoreRequest

Field	Type	Description	Required
`store`	`Store`		No

UpdateStoreResponse

Field	Type	Description	Required
`store`	`Store`		No
`status`	`Status`		No

CreateProjectRequest

Request to create a project

Field	Type	Description	Required
`name`	`string`	Name of project (required)	No

CreateProjectResponse

Response for creation of a project

ArchiveProjectRequest

Request for the archival of a project

Field	Type	Description	Required
`name`	`string`	Name of project to be archived	No

ArchiveProjectResponse

Response for archival of a project

ListProjectsRequest

Request for listing of projects

ListProjectsResponse

Response for listing of projects

Field	Type	Description	Required
`projects`	`string[]`	List of project names (archived projects are filtered out)	No

UpdateFeatureSetStatusResponse

ApplyFeatureTableRequest

Field	Type	Description	Required
`project`	`string`	Optional. Name of the Project to apply the Feature Table to. If unspecified, will apply FeatureTable to the default project.	No
`tableSpec`	`FeatureTableSpec`	Feature Table specification to apply	No

ApplyFeatureTableResponse

Field	Type	Description	Required
`table`	`FeatureTable`		No

GetFeatureTableRequest

Field	Type	Description	Required
`project`	`string`	Optional. Name of the Project to retrieve the Feature Table from. If unspecified, will apply FeatureTable to the default project.	No
`name`	`string`	Name of the FeatureTable to retrieve.	No

GetFeatureTableResponse

Field	Type	Description	Required
`table`	`FeatureTable`	The Feature Table retrieved.	No

ListFeatureTablesRequest

Field	Type	Description	Required
`filter`	`Filter`	Filter used when listing Feature Tables	No

ListFeatureTablesResponse

Field	Type	Description	Required
`tables`	`FeatureTable[]`	List of matching Feature Tables	No

DeleteFeatureTableRequest

Field	Type	Description	Required
`project`	`string`	Optional. Name of the Project to delete the Feature Table from. If unspecified, will delete FeatureTable from the default project.	No
`name`	`string`	Name of the FeatureTable to delete.	No

DeleteFeatureTableResponse

ListEntitiesRequest.Filter

Field	Type	Description	Required
`project`	`string`	Optional. Specifies the name of the project to list Entities in. It is NOT possible to provide an asterisk with a string in order to do pattern matching. If unspecified, this field will default to the default project ‘default’.	No
`labels`	`map<string, string>`	Optional. User defined metadata for entity. Entities with all matching labels will be returned.	No

ListFeaturesRequest.Filter

Field	Type	Description	Required
`labels`	`map<string, string>`	User defined metadata for feature. Features with all matching labels will be returned.	No
`entities`	`string[]`	List of entities contained within the featureSet that the feature belongs to. Only feature tables with these entities will be searched for features.	No
`project`	`string`	Name of project that the feature tables belongs to. Filtering on projects is disabled. It is NOT possible to provide an asterisk with a string in order to do pattern matching. If unspecified this field will default to the default project ‘default’.	No

ListStoresRequest.Filter

Field	Type	Description	Required
`name`	`string`	Name of desired store. Regex is not supported in this query.	No

ListFeatureTablesRequest.Filter

Field	Type	Description	Required
`project`	`string`	Optional. Specifies the name of the project to list Feature Tables in. If unspecified would list Feature Tables in the default project.	No
`labels`	`map<string, string>`	Optional. Feature Tables with all matching labels will be returned. If unspecified would list Feature Tables without filtering by labels.	No

Job

Field	Type	Description	Required
`id`	`string`	Identifier of the Job	No
`type`	`JobType`	Type of the Job	No
`status`	`JobStatus`	Current job status	No
`hash`	`string`	Deterministic hash of the Job	No
`retrieval`	`RetrievalJobMeta (oneof)`		No
`batchIngestion`	`OfflineToOnlineMeta (oneof)`		No
`streamIngestion`	`StreamToOnlineMeta (oneof)`		No

StartOfflineToOnlineIngestionJobRequest

Ingest data from offline store into online store

Field	Type	Description	Required
`project`	`string`	Feature table to ingest	No
`tableName`	`string`		No
`startDate`	`Timestamp`	Start of time range for source data from offline store	No
`endDate`	`Timestamp`	End of time range for source data from offline store	No

StartOfflineToOnlineIngestionJobResponse

Field	Type	Description	Required
`id`	`string`	Job ID assigned by Feast	No

GetHistoricalFeaturesRequest

Field	Type	Description	Required
`featureRefs`	`string[]`	List of feature references that are being retrieved	No
`entitySource`	`DataSource`	Batch DataSource that can be used to obtain entity values for historical retrieval. For each entity value, a feature value will be retrieved for that value/timestamp Only ‘BATCH*’ source types are supported. Currently only BATCHFILE source type is supported.	No
`project`	`string`	Optional field to specify project name override. If specified, uses the given project for retrieval. Overrides the projects specified in Feature References if both are specified.	No
`outputLocation`	`string`	Specifies the path in a bucket to write the exported feature data files Export to AWS S3 - s3://path/to/features Export to GCP GCS - gs://path/to/features	No
`outputFormat`	`string`	Specify format name for output, eg. parquet	No

GetHistoricalFeaturesResponse

Field	Type	Description	Required
`id`	`string`	Export Job with ID assigned by Feast	No
`outputFileUri`	`string`		No

StartStreamToOnlineIngestionJobRequest

Field	Type	Description	Required
`project`	`string`	Feature table to ingest	No
`tableName`	`string`		No

StartStreamToOnlineIngestionJobResponse

Field	Type	Description	Required
`id`	`string`	Job ID assigned by Feast	No

ListJobsRequest

Field	Type	Description	Required
`includeTerminated`	`bool`		No

ListJobsResponse

Field	Type	Description	Required
`jobs`	`Job[]`		No

GetJobRequest

Field	Type	Description	Required
`jobId`	`string`		No

GetJobResponse

Field	Type	Description	Required
`job`	`Job`		No

CancelJobRequest

Field	Type	Description	Required
`jobId`	`string`		No

CancelJobResponse

Job.RetrievalJobMeta

Field	Type	Description	Required
`outputLocation`	`string`		No

Job.OfflineToOnlineMeta

Job.StreamToOnlineMeta

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

feast.types.ValueType.Enum

Name	Description
`INVALID`
`BYTES`
`STRING`
`INT32`
`INT64`
`DOUBLE`
`FLOAT`
`BOOL`
`BYTES_LIST`
`STRING_LIST`
`INT32_LIST`
`INT64_LIST`
`DOUBLE_LIST`
`FLOAT_LIST`
`BOOL_LIST`

DataSource.SourceType

Type of Data Source.

Name	Description
`INVALID`
`BATCH_FILE`
`BATCH_BIGQUERY`
`STREAM_KAFKA`
`STREAM_KINESIS`

DataSource.SourceType

Type of Data Source.

Name	Description
`INVALID`
`BATCH_FILE`
`BATCH_BIGQUERY`
`STREAM_KAFKA`
`STREAM_KINESIS`

Store.StoreType

Name Description

INVALID

REDIS

Redis stores a FeatureRow element as a key, value pair.

The Redis data types used (https://redis.io/topics/data-types): - key: STRING - value: STRING

Encodings: - key: byte array of RedisKey (refer to feast.storage.RedisKeyV2) - value: Redis hashmap

REDIS_CLUSTER