Aurora uses Apache Thrift for representing structured data in client/server RPC protocol as well as for internal data storage. While Thrift is capable of correctly handling additions and renames of the existing members, field removals must be done carefully to ensure backwards compatibility and provide predictable deprecation cycle. This document describes general guidelines for making Thrift schema changes to the existing fields in api.thrift.
It is highly recommended to go through the Thrift: The Missing Guide first to refresh on basic Thrift schema concepts.
Every existing Thrift schema modification is unique in its requirements and must be analyzed carefully to identify its scope and expected consequences. The following checklist may help in that analysis: * Is this a new field/struct? If yes, go ahead * Is this a pure field/struct rename without any type/structure change? If yes, go ahead and rename * Anything else, read further to make sure your change is properly planned
Any time a breaking change (e.g.: field replacement or removal) is required, the following cycle must be followed:
Change is applied in a way that does not break scheduler/client with this version to
communicate with scheduler/client from vCurrent-1.
* Do not remove or rename the old field
* Add a new field as an eventual replacement of the old one and implement a dual read/write
anywhere the old field is used. If a thrift struct is mapped in the DB store make sure both columns
are marked as
* Check storage.thrift to see if
the affected struct is stored in Aurora scheduler storage. If so, it’s almost certainly also
necessary to perform a DB migration.
* Add a deprecation jira ticket into the vCurrent+1 release candidate
* Add a TODO for the deprecated field mentioning the jira ticket
Finalize the change by removing the deprecated fields from the Thrift schema. * Drop any dual read/write routines added in the previous version * Remove thrift backfilling in scheduler * Remove the deprecated Thrift field
It’s always advisable to test your changes in the local vagrant environment to build more
confidence that you change is backwards compatible. It’s easy to simulate different
client/scheduler versions by playing with
aurorabuild command. See this document