text: Thursday, May 03, 2018
  k8s, k8s everywhere:
  - Announcing early access to DigitalOcean Kubernetes
  - Introducing the Operator Framework: Building Apps on Kubernetes
  and google is slowly killing docker, by open sourcing new build image tool and container runtime:
  - Open-sourcing gVisor, a sandboxed container runtime
  - Introducing kaniko: Build container images in Kubernetes and Google Container Builder without privileges
  a good article on unprivileged container: Towards unprivileged container builds
  I think I'll just watch, tools are changing so fast in this area ...
  right now I'm more interested in graalvm:
  so, pandas for everyone?
  I didn't pay much attention to it, until I watched this talk: Scaling Uber's Elasticsearch Clusters
  they allows user input sql to query elasticsearch, a query proxy parses the query and transforms to elasticsearch syntax, (also routing, rate limiting, caching and workload management)
  the query parser mentioned should be queryparser
  Using SQL to query Kafka, MongoDB, MySQL, PostgreSQL and Redis with Presto
  this is so cool, query all different data sources in one place
  more on presto: Engineering Data Analytics with Presto and Parquet at Uber
  finally some links
  - some good sql blog posts
  - towards λ-calculus
  - A Look at MyRocks Performance
  - CPU utilization is wrong, says Netflix's Brendan Gregg
  - Tips for High Availability from Netflix
  Sunday, May 06, 2018
  reading few books at the same time (from local library):
  - Think Data Structures: Algorithms and Information Retrieval in Java
  - Effective Java (3rd Edition)
  - Designing for Scalability with Erlang/OTP: Implement Robust, Fault-Tolerant Systems
  tried few micro emacs editors:
  also
  mcedit can switch to emacs keybindings:
  $ cp /etc/mc/mc.emacs.keymap ~/.config/mc/mc.keymap
  they work almost the same, I'll stick with
  mcedit
  good introduction on gvisor: Sandboxing Docker with Google's gVisor
  Importantly, gVisor does not simply redirect application system calls through to the host kernel. Instead, gVisor implements most kernel primitives (signals, file systems, futexes, pipes, mm, etc.) and has complete system call handlers built on top of these primitives.
  and it shares same network stack with fuchsia: go/src/netstack
  so it's an os written in go for desktop, container and protentially app?
  here's a nice overview on chroot, cgroups and namespaces
  clojure has more experiment works on the
  tools.deps: add-lib
  Rich Hickey interviewed in a podcast
  looks like he's working on a new version of Datomic/codeq: Creates Datomic dbs from git repos
  I am always interested in writing a proxy:
  Hosting multiple apps on the same server — Implement a reverse proxy with Node
  I still don't have time to upgrade my lxd to
  3.0 yet
  here's a guide for the cluster mode: LXD Clusters: A Primer
  Using Kubernetes for Local Development
  it uses GoogleContainerTools/skaffold: Easy and Repeatable Kubernetes Development
  Monday, May 07, 2018
  use Paramiko to execute remote commands:
  import paramiko import os client = paramiko.SSHClient() client._policy = paramiko.WarningPolicy() client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) client.load_system_host_keys() ssh_config = paramiko.SSHConfig() with open(os.path.expanduser('~/.ssh/config')) as f: ssh_config.parse(f) host_data = ssh_config.lookup('stg-data') cfg = {'hostname': host_data['hostname'], 'username': host_data['user'], 'pkey': paramiko.ed25519key.Ed25519Key(filename=host_data['identityfile'][0])} client.connect(**cfg) stdin, stdout, stderr = client.exec_command('ls /tmp/') for line in stdout: print(line) client.close()
  above reads '~/.ssh/config
  for host ip and supports Ed25519` key
  this hackernews thread brings me back to
  eshell:
  c-u m-x eshell: open new eshell
  alias iff ido-find-file
  C-c M-b: Inserts the printed buffer name at point
  C-c M-i: Inserts the printed process name at point
  C-c M-v: Inserts an environment variable name at point
  ~/.eshell/profile for startup scripts and
  ~/.eshell/alias for alias
  for loop examples:
  for f in web1 web2 { scp info.php $f:/var/www/html/oscmax/catalog } for f in {ls -1d *} {echo $f; svn up $f} for f in {find . -name '*.java'} {mv $f (replace-regexp-in-string "\\.java$" ".scala" f)} # Those parens may also be curly braces. Paren for lisp, curly for subshell. # See how $f is used in curly brace but f is used in parens for f in db1 db2 {mysqldump -d -uuser -ppwd (concat "prefix_" f "_suffix") >(concat "/tmp/backup_" f ".sql")}
  this eshell prompt is very cool: Making eshell your own
  Thursday, May 17, 2018
  finished reading Think Data Structures: Algorithms and Information Retrieval in Java, it's good and I'm planning to implement examples in clojure
  still reading Effective Java (3rd Edition), it's such a great book. I'm have to buy a copy.
  borrowed another book from local library:
  another book but not computer related: History of Pictures: From the Cave to the Computer Screen, about drawing, painting, lens and photography, great book.
  Saturday, May 26, 2018
  it's GDPR day, mozilla has a good article on it: 13 things to know about the GDPR
  I think it's a good thing, it could help us to understand what a free service really is
  while reading this interesting tool: dgsh - directed graph shell
  I discovered couple more:
  Apache Flink 1.5.0 Release Announcement
  I've been reading some flink documentations, now it's time to upgrade: Apache Flink 1.5-SNAPSHOT Documentation
  here's an article on different stream processing system: Spark Streaming vs Flink vs Storm vs Kafka Streams vs Samza : Choose Your Stream Processing Framework
  I saw only good things on flink
  I read last chapter on Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale, about its stream api
  but I need to see more examples ...
  some about reasonml
  Sunday, May 27, 2018
  I hate scrum, but the question is if not scrum then what?
  Reality Driven Development: Fixing Project Management in Software
  Kanban is an ideal software project methodology when you remove accountability concerns from the equation. The best developers I know are obsessed with solving problems. If they are pulled in too many directions, when the end of the day comes they'll just turn their brain off and mentally check out for the evening.
  If they are focused on a single problem...they will obsess about that single problem after hours. They will sleep on it. They'll be distracted while watching TV. They'll run to the computer at 10pm to test an idea that they had about the problem.
  As a business owner, you want those extra mental hours spent solving your problem. At the same time, you have to identify when you have people who do this for you and insistently make them take extra time off every now and then. Whether for their family, their kids or their health the last thing you want to do is burn out an asset like that by letting a person push themselves hard enough to create problems outside of work.
  a little bit scary for me (as a developer)
  at the end author suggested kanban and light pairing, which is make sense to me
  this is also good, Great PMs don’t spend their time on solutions
  spend more time to define the problem.
  another good one: You’re Doing Scrum Wrong, and Here’s How to Fix It
  it introduces spike (from extreme programming I believe), which is much much better than sprint zero
  Spot A Fake Agile Team In Under 7 Minutes
  you asks two questions:
  - How is feedback gathered?
  - How much budgets is available for changes?
  make sense, it's all about the iteration
  I actually learned few things by reading all these articles recently:
  first one, agile manifesto said:
  Individuals and interactions over processes and tools
  so if scrum doesn't work out for your team, drop it, don't force to fit into the framework
  the other thing is deadline
  you can try search the scrum guide, you won't able to find deadline in it.
  no deadline is too good to be true, so how do scrum deal with it?
  there's only one way - cut features, which means even requirements can be changed and should adapt agile
  however it's same as no deadline, too good to be true.
  so any scrum guide that doesn't cover how to deal with deadline is bullshit.
  finally, the best comes from this comment: Agile often allows no time for doing it right, leading to high levels of technic...
  Agile is also totally unsuited for innovation. You will never get a Xerox Parc or SRI with agile. You can only make what you already know to make with agile.
  that sums up my thoughts on scrum
  I'm following these guides for some experiment:
  - Hadoop 3 Single-Node Install Guide
  - Using SQL to query Kafka, MongoDB, MySQL, PostgreSQL and Redis with Presto
  ideas are from uber's blogs:
  - Engineering Data Analytics with Presto and Parquet at Uber
  - Scaling Uber’s Hadoop Distributed File System for Growth
  also evaluating linkedin/pinot: A realtime distributed OLAP datastore
  and compare it with clickhouse and druid: Comparison of the Open Source OLAP Systems for Big Data: ClickHouse, Druid and Pinot
  learned something new with emacs:
  m-x ffap: open file at point, (short name for
  find-file-at-point)
  ibuffer
  filters:
  /m: by major mode
  /n: by name
  /c: by content
  /f: by filename
  once filter group created, use
  /g to name it,
  // to clear filters
  sort:
  s a: by alphabet
  s f: by filename
  s v: by last viewed time
  s s: by size
  s i: invert sorting
  misc:
  =: compare buffer and file content
  o/
  c-o: open, and whether to switch point to it
  m-p: rotate filter groups
  bookmark
  c-x r m: set bookmark at point
  c-x r b: jump to bookmark
  c-x r l: open bookmark list
  mark
  c-spc c-spc: to place a mark
  c-u c-spc: return to a mark
  c-x c-x: swap with last mark
  occur
  m-s o: occur
  m-s h .: highlight symbol at point
  m-s p .: highlight phase
  m-s h l: highlight line
  m-s h r: highlight regex
  m-s h u: reset highlight
  m-g n/
  m-g p: navigate matches
  in the
  *Occur* buffer:
  o: goto matches
  g: refresh
  q: quit
  repeat
  c-x z: repeat previous command (like
  .in vim),
  z: continue to repeat
  rgrep
  tab/
  s-tab: next / previous match
  {/
  }: previous / next matched file
  spc/
  s-spc: sroll down / up
  g: refresh
  q: quit
  dired
  i: insert sub-directory to current buffer (saves you lots of buffers)
  c-x r w [reg]: save window
  c-x r j [reg]: jump to register
  c-x r s [reg]: copy region
  c-x r i [reg]: insert copied region
  c-x r r [reg]: save retangle
  c-x r spc [reg]: save point
  Tuesday, May 29, 2018
  I compiled with Mitsuharu Yamamoto's fork
  after compiled, it shows two errors:
  spinner.elerror, solved by changing
  (timer (timer-create) :read-only nil)to
  (timer (timer-create) :read-only nil)
  Wrong type argument: stringp, package-desc(solved with
  M-: (byte-recompile-directory package-user-dir nil 'force))
  also change
  command key as
  meta:
  (setq mac-command-modifier 'meta)
  Blog Archives
  Older Entries
  -