text: Build an API with Kotlin on Google Cloud Platform
  In this tutorial you will learn how to build a server side API using Kotlin and Ktor that you can host on Google Cloud Platform and use with your Android app.
  Version
  - Kotlin 1.3, Other, Android Studio 3.5
  Google Cloud Platform has made building Kotlin-based server applications more accessible than ever. If you’ve always done mobile development, creating a server application in Kotlin for your Android app might look intimidating. Fear not, thanks to Google and Google Cloud Platform, you’ll be creating APIs in no time!
  Picture this situation: the boss gives you an Android application for a conference. They also give you a Google Sheets document with the list of talks that you need to show, and ask you to create an API for the mobile app.
  Since you’re a willing coder, you’ll of course take on this project with vim and vigor. As you build out this theoretical API, you’ll learn how to:
  - Configure Google Cloud Console and AppEngine
  - Create a server-side application, using Ktor and Kotlin
  - Access APIs for Google services, like Spreadsheets
  - Use data persistency using DataStore
  - Deploy to a scalable service
  Getting Started
  Download the starter project from the link at the top or bottom of this tutorial. In there, you can find a module with an Android app, a common module for shared code, and a server module for the API. In this tutorial, you’ll be working on the server module.
  Before moving into the code part, you need to set up Google Cloud and create a new project. If this is your first time working with it, make sure you follow these steps in detail. It can be a long process, so grab a drink and get comfortable. Fortunately, you only have to do this once! :]
  Setting up Google Cloud
  To manage Google Cloud, you need to have an internet browser and be signed in with a Google account. If you don’t have a Google account already, make sure to create one before continuing.
  Creating a Google Cloud Project
  First, you need to create a new Google Cloud project. To do so navigate to the following URL:
  If it’s your first time using Google Cloud, you’ll have to accept their Terms of Service and select a country.
  Once you’ve accepted the terms, you can name the project something meaningful. Name it “RayConf”:
  Then click Create, which will forward you to the project’s dashboard. There you can see a summary of the resources used.
  Creating an AppEngine Application
  Now that you have a project, you need to create a new AppEngine Application. AppEngine is one of the many services that Google Cloud provides. It removes all the complications of creating a scalable service. You’ll be deploying the API there.
  A project can have many Server Applications, but for this application, you only need one. Click the hamburger menu in the top-left corner, scroll down to the Compute section and click App Engine.
  You will be taken to the AppEngine Dashboard view, with the below card already in place.
  Select Create Application. Next, the setup process will ask you to select a region and zone. For this tutorial, you can use the defaults and select Create App.
  Next, you need to select your language and environment. For the language, you have to select
  Java although you are going to use Kotlin, it will run on the Java Virtual Machine (JVM). You can leave the Environment as Standard and select Next.
  You will see a confirmation message on the screen.
  Installing Google Cloud
  Now, follow the instructions to install by selecting the Download the SDK button. Once installed, you can verify that it works by running the following command in the terminal:
  gcloud --version
  You may need to restart the IDE for it to recognize the changes.
  Logging into Google Cloud
  Brilliant, now that you have the SDK installed, you need to log in. To do so, run the following command in the terminal:
  gcloud auth login
  The browser will open and ask you to authorize the SDK with your Google account. Make sure this is the same account under which you created the AppEngine Project.
  Once logged in, the project ID will be set to the only project you have on your account.
  gcloud projects list
  Look for “rayconf” in the list and copy the complete PROJECT_ID.
  Once you have the PROJECT_ID, you can switch to the project by running the following command:
  gcloud config set project PROJECT_ID
  Also, to make sure you have the latest version installed, run the following command:
  gcloud components update
  Enabling Google Sheets API
  Now, you want to use the Google Sheets document your boss gave you to generate the API. Therefore, you have to permit AppEngine to access Sheets. To do so, you need first to enable the API here:
  Authorizing Service Account
  Once the API is enabled, you need to create an account for the server to use. These accounts, called service accounts, are useful when there is no option for a user to log in, as is the case in the cloud. Go to the following URL to manage service accounts:
  - and click on Create service account.
  Give it a meaningful name and select Create.
  Choose Editor as the role so you can edit resources, such as persistent data. No spoilers, but it’ll be handy later.
  Select Continue. On the next screen, you have the option to grant extra permissions and to create a key. It’s safe to ignore the first part, but you need to create a key so the server can authenticate itself.
  Select Create key, make sure you have JSON ticked, and select the Create button.
  Doing so generates the key and downloads a JSON file. This key is important as it grants your app access to resources, so make sure you keep it safe. To install it in the app, rename it to credential.json and place it in the following folder:
  The sample project has a convenience object named AppCredentials. It loads the key file into memory when needed.
  I’m sure you will be glad to hear that this was the last thing you need to set up on Google Cloud.
  Now it’s time to get to the fun part, actual code! :]
  Creating your First Endpoint
  You are going to create an endpoint that returns the classic
  Hello World!message.
  Start by adding a new Kotlin file to the server module and name it RayConfApp.kt:
  Inside this file, create an extension function named
  mainon
  Applicationclass:
  import io.ktor.application.Application import io.ktor.application.call import io.ktor.response.respond import io.ktor.routing.get import io.ktor.routing.routing fun Application.main() { routing { // 1 get("/") { // 2 call.respond("Hello World!!") // 3 } } }Note:You can learn more about extension functions here: Programming in Kotlin · Extension
  This function, once registered, is run when the server creates the Application. At which point, you will be:
  - Defining a group of routes for the server
  - Letting the server Application know that you want to handle a GET request on the root path
  - Then, every time that request is made you respond to the call with a given response
  You can now run the app by executing the following command in the terminal:
  ./gradlew appengineRun
  After some time the server will be accessible at the following URL:
  Open it in a browser to see the result.Note: To stop the server, use the keyboard shortcut Control + C.
  The main function is registered with the application in the application.conf file:
  Ktor provides a great DSL (Domain Specific Language) for defining clear and descriptive servers. You’ve named the function
  main, but you can use any name and define different ones. This way, it’s easy to structure an application into separate parts for easy management.Note: Always remember to update the
  application.confwhen adding new ones.
  Defining a Server Endpoint for Talks
  It’s time to create something a bit more useful than a “Hello World!” example. The common module contains the model that the Android app is using. Since you are also writing the server application in Kotlin, you can use it in the server-side. Isn’t that efficient?
  Start by adding another
  get("talks")block, right below the existing
  get("/")block inside RayConfApp.kt file:
  fun Application.main() { routing { get("/") { ... } get("talks") { // Add response here } } }
  Next, you need a list of events to return. For now, you can add it on the same file:
  import com.raywenderlich.common.AgendaEntry ... private val talks = listOf( AgendaEntry( id = 0, title = "This is the first talk", date = "28/09/2019", startTime = "09:00:00", endTime = "10:00:00", description = "Description for something very interesting, we hope.", speaker = "TBC" ) )
  Now, you need to return a String that represents the contents of this list. For this, you need a way to convert the model into a String. Kotlin provides a simple way to do such serialization with kotlinx.serialization.
  To set it up in the project, you will need to first add the plugin’s classpath. To do so navigate to build.gradle file at the root of the project. Then replace
  // Todo: Add kotlin-serialization plugin's classpath herewith below:
  classpath "org.jetbrains.kotlin:kotlin-serialization:$kotlin_version"
  Next, navigate to build.gradle file for the common module and apply the kotlin-serialization plugin by replacing
  // Todo: Add kotlin-serialization plugin herewith below:
  apply plugin: 'kotlinx-serialization'
  Lastly, add the kotlin-serialization dependency by replacing
  // Todo: Add kotlin-serialization dependency herewith below:
  implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.13.0"
  All done, sync the project with Gradle to download all dependencies and configure plugins.
  After adding the kotlinx serialization dependency to the project, adding the
  @Serializableannotation on
  AgendaEntryclass is all that is needed:
  import kotlinx.serialization.Serializable @Serializable // <-- added data class AgendaEntry( ... )
  Then, hide the fact that you’re using Kotlin serialization by adding parser functions inside the AgendaEntry.kt file:
  import kotlinx.serialization.json.Json import kotlinx.serialization.list fun AgendaEntry.toJsonString(): String = Json.stringify(AgendaEntry.serializer(), this) fun List<AgendaEntry>.toJsonString(): String = Json.stringify(AgendaEntry.serializer().list, this)
  Hiding away your
  toJsonString() implementations like this is helpful for maintenance. It means that future changes to the serialization won't affect other parts of the code.
  The first function uses the generated serializer,
  AgendaEntry.serializer(), to convert the entity into a string. The second one does the same but using the equivalent for lists:
  AgendaEntry.serializer().list. More information about this can be found in the official documentation here.
  Now you can go back to the
  main function inside the RayConfApp.kt file. In there, add the code for the response, inside the
  get("talks") block, making sure all imports are resolved:
  call.respondText(contentType = ContentType.Application.Json) { talks.toJsonString() }
  Here, you are telling Ktor:
  - To respond with some text
  - To set the content type to
  application/json
  - And, finally, the lambda at the end returns a string which is a representation of the list of talks
  Time to see it in action. Run the server again using
  ./gradlew appengineRun. Once deployed locally, verify in your browser that the result from has a list with a single entry:
  Deploying to Google Cloud
  Now that you have something non-trivial, you can take it to the next level. You can deploy it to an actual server in the cloud! :]
  This is where all the hard work you did setting Google Cloud pays off. Run the below command in the terminal of the IDE.
  ./gradlew appengineDeploy
  When it starts building, you should see something similar to this:
  The output tells you where the SDK deploys the app. So, once it’s finished building, visit
  https://[your_appengine_project_name].appspot.com/talks and see that it shows the same list with one entry on your browser. Remember to replace the project name with yours.
  Next, go to MainActivity.kt and replace
  [your_appengine_project_name] with the name that you got from the deployment. It doesn't change on each deployment, so this is the only time that you’ll have to change it.
  Now the Android app is ready to start consuming the service. Run the app by selecting the app configuration and clicking the play button next to it:
  The app runs and displays just one event:
  Adding Dynamic Data with Google Drive
  Now, it's time to load some data from Google Sheets.
  Reading Data From a Spreadsheet
  Create a new Kotlin class named SpreadSheetDataSource, right next to RayConfApp.kt. This class is going to be in charge of loading data from a Google Sheet.
  In this file, add the below code:
  class SpreadSheetDataSource { // 1 private val transport = GoogleNetHttpTransport.newTrustedTransport() // 2 private val jacksonFactory = JacksonFactory.getDefaultInstance() // 3 private val sheets = Sheets.Builder(transport, jacksonFactory, AppCredentials.local).build() // 4 private val getAll = sheets.spreadsheets() .values().get("10mOp9WSGCZbMmlreLwy0GJiMBeTN_t05TaXXmFmBnVI", "A:G") // 5 fun listAll() = getAll.execute().getValues() }
  transportprovides the Google API the means to generate and execute network calls.
  jacksonFactoryprovides the Google API the means to deserialize entities.
  - Create a new Sheets Builder with required arguments such as the credentials that you included in the resources as
  credential.json.
  - Using
  sheetsinstance to define the
  getAllquery. It fetches all the available rows for the first six columns:
  A:G. The first parameter is the ID of the Spreadsheet.
  listAll()method executes the query when asked and returns the values.
  You may have noticed that the return type for that last function is of type:
  (Mutable)List<(Mutable)List<Any!>!>!
  That scary sausage of code means that it returns a list containing lists of objects of
  Any type. Not extremely useful. So, you’ll have to convert it into something to present to the user.
  Combining Headers and Rows
  Start by creating a function inside SpreadSheetDataSource.kt to convert the raw columns to a list of maps:
  private fun List<List<Any>>.toRawEntries(): List<Map<String, Any>> { val headers = first().map { it as String } // 1 val rows = drop(1) // 2 return rows.map { row -> headers.zip(row).toMap() } // 3 }
  The
  toRawEntries function doesn't change the state of the data source so you can define it in the same file, as a top-level function.
  In this function, you are:
  - Extracting the first list and casting the values to
  Strings.
  - Then, you drop the first row to get the data rows.
  - Here you are converting each row to a map. The
  zipfunction creates a
  List<Pair<String, Any>>.
  zippairs the elements with the same index. Finally, there is a convenience function called
  toMap()that converts a list of pairs into a
  Map.
  Building Agenda Entries
  Next, you need a function to convert a
  Map into a
  AgendaEntry.
  private fun Map<String, Any>.toAgendaEntry() = AgendaEntry( id = longFor("Id"), title = stringFor("Title"), date = stringFor("Date"), startTime = stringFor("Start Time"), endTime = stringFor("End Time"), description = stringFor("Description"), speaker = stringFor("Speaker") ) private fun Map<String, Any>.stringFor(key: String) = (this[key] as? String) ?: "" private fun Map<String, Any>.longFor(key: String) = stringFor(key).toLongOrNull() ?: -1
  stringFor(key) looks up a value in the receiver
  Map and tries to cast it to a
  String using
  as?
  . This conditional cast operator returns
  null if the cast is not successful, in which case you return an empty String.
  longFor(key) uses the previous function to look up a value, and after
  toLongOrNull(), tries to parse the string into a Long. A null result means it can’t parse it into a Long, so you return a default value of
  -1.
  toAgendaEntry() uses these two functions to build a
  AgendaEntry matching the headers from the following spreadsheet:
  Putting the Transformations Together
  Now, update
  listAll to return a list of
  AgendaEntry entities:
  fun listAll(): List<AgendaEntry> = getAll.execute().getValues() .toRawEntries() .map { it.toAgendaEntry() }
  The return value now is a list of
  AgendaEntry.
  Integrating the SpreadSheetDataSource
  Now, go to the
  main() function inside RayConfApp.kt file and create a property named
  entries with the data source:
  val entries = SpreadSheetDataSource()
  Next replace
  talks.toJsonString() with
  entries.listAll().toJsonString() inside
  get("talks") block:
  get("talks") { call.respondText(contentType = ContentType.Application.Json) { entries.listAll().toJsonString() // <--- replaced here } }
  Run the server locally using the command
  ./gradlew appengineRun in the terminal. In the browser, the result from now has all the entries from the Spreadsheet.
  Now, deploy the server to the cloud using
  ./gradlew appengineDeploy. Once deployed, all the events are also visible on the Android app when relaunched/refreshed.
  - Create a copy you can edit.
  - Make the copy public.
  - Change the ID of the spreadsheet inside
  SpreadSheetDataSourceto use the ID from your copy. You can find the ID on the URL of the document, as the segment after
  /d/
  Caching Response
  One thing you may have noticed is that the request takes a long time to return, anywhere between 300ms and 2s. Such a delay is understandable since the server is talking to Google Drive each time. Google Cloud provides several ways to persist data, making it more readily available. For this API, you are going to use Data Storage which provides basic key-value storage. You can see other storage options here.
  Mapping Entities
  Before writing a Local data source, you need a couple of mapper functions to convert from and to
  Entity instances. So, create a file named LocalDataSource.kt under the server module and add the following code:
  import com.google.appengine.api.datastore.* import com.raywenderlich.common.AgendaEntry private const val AGENDA_ENTRY_NAME = "AgendaEntry" private fun AgendaEntry.toEntity() = Entity(AGENDA_ENTRY_NAME, id).apply { setProperty("id", id) setProperty("title", title) setProperty("date", date) setProperty("startTime", startTime) setProperty("endTime", endTime) setProperty("description", description) setProperty("speaker", speaker) setProperty("votes", votes) setProperty("updated", updated) } private fun Entity.toEntry() = AgendaEntry( id = getProperty("id") as Long, title = getProperty("title") as String, date = getProperty("date") as String, startTime = getProperty("startTime") as String, endTime = getProperty("endTime") as String, description = getProperty("description") as String, speaker = getProperty("speaker") as String, votes = getProperty("votes") as Long, updated = getProperty("updated") as Long )
  Entity is a type from
  DataStore. It is similar to a
  Map. You can read and write values with the following two methods:
  getProperty(key: String): Any?tries to find a value for the provided key or returns null if not found.
  setProperty(key: String, value: Any?)creates or updates a value for a given key.
  For the
  toEntity() mapper, you are creating a new
  Entity with
  "AgendaEntry" as the name and the
  AgendaEntry's
  id as the ID for the
  Entity. The ID is used to find and/or update entities. Then, you set the properties with the relevant keys.
  In
  toEntry(), you do the reverse. Using the same keys used for the previous function, you read the relevant value and cast it to either a
  String or a
  Long.
  Creating a Local Data Source
  Now that you can transform your application models into DataStore Entities, and vice versa, you can start building a local data source. Add the class
  LocalDataSource inside the LocalDataSource.kt file:
  class LocalDataSource { private val dataStore = DatastoreServiceFactory.getDatastoreService() private val agendaEntryQuery = dataStore.prepare(Query(AGENDA_ENTRY_NAME)) fun listAll(): List<AgendaEntry> = agendaEntryQuery.asList(FetchOptions.Builder.withDefaults()).map { it.toEntry() } fun save(entries: List<AgendaEntry>): List<Key> { entries.map { it.toEntity() }.also { return dataStore.put(it) } } }
  The two properties in this component are a
  DataStore instance and a query to get all values for entities with the name "AgendaEntry". There are also two entry points:
  listAll()is similar to what you had in
  SpreadSheetDataSource, but this time you are executing
  agendaEntryQueryon
  dataStoreas a list and mapping it to Entries using
  toEntry.
  save(entries)lets you put a list of entries into the
  dataStore. The Data Store overrides any entities which have the same ID.
  Composing Data Sources
  Now you have a new data source to read and write instances of
  AgendaEntry in the local cache. So now you need to combine it with your other data store and do some "cache balancing".
  Create a file named AgendaEntryRepository.kt under the server module and add the following code:
  import com.raywenderlich.common.AgendaEntry private const val HOUR = 3_600_000L class AgendaEntryRepository { private val spreadSheetDataSource = SpreadSheetDataSource() private val cacheDataSource = LocalDataSource() private val cachePeriod: Long = HOUR fun listAll(): List<AgendaEntry> = cacheDataSource.listAll().let { cached -> cached.takeUnless { it.requiresUpdate(cachePeriod) } ?: spreadSheetDataSource.listAll() .also { entries -> cacheDataSource.save(entries) } } } private fun List<AgendaEntry>.requiresUpdate(cachePeriod: Long) = isEmpty() || any { System.currentTimeMillis() - it.updated > cachePeriod }
  In this component, you have three properties: two for the different data sources and a third one for how long the entities are kept in memory.
  listAll() first tries to retrieve the data from the LocalDataSource. Then, you return the cached entities unless they need to be reloaded from the spreadsheet. The decision to load data from the spreadsheet comes down to whether the cache has no entries, or whether the last time they were loaded was over an hour ago.
  If an update is required, you handle it using the Elvis operator. There, you fetch the values from the spreadsheet and tell the
  cacheDataSource to save the values for next time.
  Integrating the Cache
  Go to the
  main() function under the RayConfApp.kt file and instead of using
  SpreadSheetDataSource, use
  AgendaEntryRepository.
  val entries = AgendaEntryRepository()
  Deploy the server again by executing
  ./gradlew appengineDeploy in the terminal, and you will see that the first request takes about 2-3 seconds, but any consecutive calls take around 10-20ms. That's a 10x improvement!
  Implementing Votes
  A read-only API is not the most interesting, as you could replace it with a static JSON file somewhere in the cloud and no one would know the difference.
  Next, you are going to add the option to increment votes for a given talk, so you know which one is more popular.
  Updating LocalDataStore to save votes
  Go to
  LocalDataSource class and add the following methods:
  fun vote(id: Long) = find(byId = id)?.also { it.incrementVotes() } private fun find(byId: Long): AgendaEntry? = try { dataStore.get(KeyFactory.createKey(AGENDA_ENTRY_NAME, byId)).toEntry() } catch (e: EntityNotFoundException) { null } private fun AgendaEntry.incrementVotes() { copy(votes = votes + 1).toEntity().also { dataStore.put(it) } }
  The private method
  find(byId: Long) looks for an entity in
  dataStore and converts it to an
  AgendaEntry. It returns
  null if the
  DataStore throws an exception because it couldn't find the expected value.
  In the other private method,
  incrementVotes() you make a copy using the copy constructor, incrementing the votes.
  And, finally,
  vote() combines these two, returning the mutated
  AgendaEntry if successful or
  null otherwise.
  Creating the Voting Endpoint
  The main function doesn't have direct access to the
  LocalDataSource class, so you’ll have to add a new method to the
  AgendaEntryRepository class to be able to vote for an entry.
  fun vote(id: Long) = cacheDataSource.vote(id)
  Now you have everything you need to create the relevant vote endpoint. Go to the
  main function and add a new route in the
  routing block:
  post("vote") { // 1 when (val id = call.request.queryParameters["id"]?.toLong()) { // 2 null -> call.respond(HttpStatusCode.BadRequest, "Missing id parameter") // 3 else -> when (entries.vote(id)) { // 4 null -> call.respond(HttpStatusCode.NotFound, "No entry found for id: $id") // 5 else -> call.respondText("Added vote for id: $id") } } }
  - Retriev the
  idURL parameter and parse it to a
  Long.
  - If there is no ID provided, you respond with a Bad Request error.
  - If you have a valid ID, you place the vote for the user.
  - If the vote didn't happen, you respond with a Not Found error.
  - Otherwise, return a success message.
  This time, to see this last change in action you’ll have to deploy to the cloud again. Open up the Android app and you’ll be able to see how you can now vote for talks:
  Storing Votes After Updates
  When the server updates the cache from the spreadsheet, the vote counts will be lost. The loss occurs because the entries that you get from the spreadsheet have no information about the votes.
  One way to replicate this behavior is to temporarily deploy a version of the server with the
  cachePeriod inside
  AgendaEntryRepository set to
  0:
  The vote count remains at zero, as the server overwrites it when updating.
  One way to fix this is to make sure you are keeping the votes when updating the cache from the spreadsheet. Add the following function to the
  AgendaEntryRepository.kt file to do this:
  private fun List<AgendaEntry>.updateVotes(old: List<AgendaEntry>): List<AgendaEntry> { // 1 val votes = old.map { it.id to it.votes }.toMap() // 2 return map { entry -> votes[entry.id]?.let { entry.copy(votes = it) } ?: entry } }
  - First, you create a Map of IDs to votes from the old list of entries.
  - Then, if there are votes for a given entry, you create a copy with the votes; otherwise you can just return the current entry.
  Add a call to this function by updating the
  listAll() function in the AgendaEntryRepository class, just before the spot where you save the entries into the dataStore, so the server will persist the votes:
  fun listAll(): List<AgendaEntry> = cacheDataSource.listAll().let { cached -> cached.takeUnless { it.requiresUpdate(cachePeriod) } ?: spreadSheetDataSource.listAll() .updateVotes(cached) // <-- added call to new function .also { entries -> cacheDataSource.save(entries) } }
  Deploy the server again, keeping
  cachePeriod as
  0. The mobile app now increments the votes.
  Now that you have verified that it works, you can restore the one hour
  cachePeriod. The server code is ready for consumers.
  Where to Go From Here?
  You can download the final project at the top or bottom of this tutorial.
  You can also find lots of examples to learn from at Ktor’s collection of samples. Our video course Server-Side Kotlin with Ktor covers using the Ktor framework to build a Kotlin web app and API, and shows how to deploy the app to Heroku and also run the app in a Docker container.
  As a challenge, let's say that your event is super-successful, so now your company has the money to have a multi-day conference. You need to add multi-day support in the app.
  How would you modify both the server-side and the mobile app to support such changes? Leave your solutions in the comments below!
  I hope you enjoyed the tutorial, and please leave any questions or comments below!