How to create custom entity in Drupal 8
I'm working on Entity API documentation here: https://drupal.org/developing/api/entity.

The question is too big to answer in detail, and it depends on whether you need content or config entities. Assuming you want content entities (stored in database, can be fieldable/revisionable/..., details in the link above), you need to do something like this, the documentation linked above will eventually cover each step, including tutorials, right now, it only covers the first steps.


Provide an entity class and add the necessary annotations to expose it as an entity.
Provide the necessary table(s), you don't have to change much if you already have your 7.x schema, but you want to add a uuid and possibly langcode column if your entity can have a language/be translatable. See the node example linked in step if you want to make some base fields (properties) translatable or revisionable, you currently need to provide the necessary tables yourself.
Provide the base field definitions in your baseFieldDefinitions() method.. Note that the way this works will change in https://drupal.org/node/2047229 (from arrays to classes).
Implement the necessary pre/post Save/load/Delete methods if you have custom logic that needs to happen e.g. before saving an entity (like updating the changed timestamp).
Implement an access controller if you have non-trivial access logic and override the checkAccess() and checkCreateAccess() methods or specify the admin_permission in your annotation if you just have a single permission that allows full access.
It is recommended although not technically required to provide an interface for your entity class (e.g. NodeInterface for Node) that includes useful (usually get/set/isSomething()) methods for your base fields, so that you can type hint on the interface and get auto-complete for those methods.


Depending on what you need, there will also be further steps, like defining the necessary forms, possibly a render controller and so on, but those should probably be separate questions if you need more information.

Notes:


Config entities are basically similar, except you need to use the config storage controller (https://drupal.org/node/2119905 will make this easier), you don't need to define table schemas (obviously) but you need to provide a config schema file, see e.g. those for taxonomy vocabularies or node types.
Content entities don't have their base fields defined as public or protected properties, instead, they are automatically managed in the values and fields properties. Access them either through get()/set() or the corresponding magic methods. ($node->title is equal to $node->get('title')).
Help on the documentation there would be very welcome. I've created a basic outline for the content/structure I'd like to have, so just create a new page and get started on e.g.  I'm also interested in having simplified tutorials, for config entities, there's one here already that I will move over to the entity documentation when ready: https://drupal.org/node/1809494
Core has quite a large number of content entity implementations, if you want to see working examples: Node, User, Term, Comment, CustomBlock, ... as the article linked in step 1 explains, you can always find the entity types provided by a module in the Entity namespaace.
@fago just gave a presentation about Entity API at the DrupalCamp Vienna this weekend, covers a lot of what I wrote here and more.