This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these
This Week I Learned
TsundokuTsundoku (積ん読) is acquiring reading materials but letting them pile up in one's home without reading them. The term originated in the Meiji era (1868–1912) as Japanese slang. It combines elements of tsunde-oku (積んでおく, to pile things up ready for later and leave) and dokusho (読書, reading books). It is also used to refer to books ready for reading later when they are on a bookshelf. As currently written, the word combines the characters for "pile up" (積) and the character for "read" (読).
OK, so I have a bunch of books that I bought years ago and never finished. Some of these I never started. So what I decided to do is take 1 hour each day before going to work, grab 3 books and read 20 minutes per book. I also have some books in the office, I am reading 20 minutes each time when I go to the office, this is currently twice a week. The book I am reading in the office is Pragmatic Thinking and Learning, I am 40% done with that book
I already finished the book War Of Art, am 25% done with Tools of Titans and just started Python Tricks
Here is a pic of the books
The books in that picture are the following
Tools of Titans: The Tactics, Routines, and Habits of Billionaires, Icons, and World-Class Performers (started)
Tribe of Mentors: Short Life Advice from the Best in the World
War Of Art (finished)
Wired To Eat
Python Tricks the book: A Buffet of Awesome Python Features (started)
Fluent Python: Clear, Concise, and Effective Programming
Book of Five Rings
How Linux Works, 2nd Edition: What Every Superuser Should Know
I continued my DevOps training and like last week, I decided to put my note at the end of this post, this way I can easily look it up again
This Week I Tweeted
Three ‘Godfathers of Deep Learning’ Selected for Turing AwardThree computer scientists who laid the foundations for many of the recent advances in artificial intelligence are being honored with this year’s Turing Award, considered the field’s highest accolade.
Geoff Hinton, an emeritus professor at the University of Toronto and a senior researcher at Alphabet Inc.’s Google Brain, Yann LeCun, a professor at New York University and the chief AI scientist at Facebook Inc., and Yoshua Bengio, a professor at the University of Montreal as well as co-founder of AI company Element AI Inc., will share this year’s award, which is given annually by the Association for Computing Machinery.
The three winners will split a $1 million prize that comes with the award, which is currently underwritten by Google
For a list of previous winners see here: https://en.wikipedia.org/wiki/Turing_Awardhttps://en.wikipedia.org/wiki/Turing_Award
Was MongoDB Ever the Right Choice?
I was reading a post recently about Red Hat removing MongoDB support from Satellite (and yes, some folks say it is because of the license changes). It made me think how often over the last few years I’ve seen post after angry post about how terrible MongoDB is and how no one should ever use. However, in all this time, MongoDB has become a much more mature product. So what happened? Did all of the hate truly come from mistakes made in the early implementation/marketing of MongoDB? Or is the problem that people are blaming MongoDB for their own lack of efforts when evaluating if it was a good fit?
Read and find out.....
Australia Is Drifting So Fast GPS Can't Keep Up
Australia is not quite where you think it is. The continent has shifted by 4.9 feet since the last adjustment was made to GPS coordinates in 1994, reports the New York Times.
A significant correction must be made by the end of the year for navigation technology to keep working smoothly.
This is interesting, I about drift because undersea cables will break, but never had an idea that it was this fast that they have to adjust GPS
Boston Dynamics’ new robot stacks boxes
It’s a “mobile manipulation robot” designed for the logistics sector. It can autonomously stack and unstack boxes onto and off pallets, and shift them onto conveyor belts. It uses an onboard vision system to track which objects go where, and to judge how to grasp and place each box. It uses a robotic technique called “force control” to nestle each box up against its neighbors. It can handle weights of up to 15 kilograms (33 pounds.)
And that is how it all starts... wondering when they will rename the company to SkyNet :-)
Some cool stuff you might enjoy
Awesome NewslettersA curated list of awesome newsletters.
Inspired by the awesome-* trend on GitHub.
There is something here for everyone
The most populous cities in the world from 1500 to 2018
This is amazing, make sure to watch the whole animation..it's like a race....
Some DevOps notes
This is for me.. not for you.... but feel free to take a look...although it won't probably make a lot of sense if you see these terms for the first time
Top 10 books from 10 to number 1
Visible Ops
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation
Release It!: Design and Deploy Production-Ready Software
Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale
Lean Software Development: An Agile Toolkit
Web Operations: Keeping the Data on Time
The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services
The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations
Leading the Transformation: Applying Agile and DevOps Principles at Scale
The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win
Devopsweekly.com mailinglist
Impedance mismatch between teams
Blameless postmortems
---------------------------------------
It's a meeting
within 48 hours of incident
run by a 3rd party
Have everything in a timeline. Not here to blame a person but making sure this doesn't happen again
Trasnparent Uptime
-------------------------
Admit failure
Sound like a human.. no doublespeak
Have a communication channel
Be authentic
Trust blockers
--------------------
Lack of Context
Conflicting goals
Open it Up
-----------------------
Chat rooms
Wiki pages
Source code (to read)
Infrastructure
Monitoring Tools
Ticket Tracker
Chatops
USe something like chat or teams instead of emails
Shared responsibility
-------------------------------
Feedback
Automation
Team culture
No Silos
Shadow IT/Bimodal IT
Groups deliberate bypassing processes to get things done
Devs responsibe for failed deployment after code checkin
Changing your ways of how you do work.... might be hard..but it will pay off
Conway's law
----------------------------
organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.
Eric S. Raymond, an open-source advocate, restated Conway's law in The New Hacker's Dictionary
James O. Coplien and Neil B. Harrison stated:
If the parts of an organization (e.g., teams, departments, or subdivisions) do not closely reflect the essential parts of the product, or if the relationship between organizations do not reflect the relationships between product parts, then the project will be in trouble ... Therefore: Make sure the organization is compatible with the product architecture.
Westrum Model
--------------------------------
Pathological (Power Oriented)
Bureaucratic (Rule Oriented)
Generative (Performance Oriented)
Management Best Practices
-----------------------------------
Independent, cross-functional teams
People first
Agile, lean processes
Kaizen
---------------------------
Small change aka continues improvement
Good processes bring good results
Gemba (Go see for yourself).. (the real pace or Locus).. meaning look at the code or the system or factory floor
Speak with data, manage by facts (scientific method)
Take action to contain and correct root causes (The 5 whys, people don't fail processes do)
Work as a team
Kaizen is everubody's business
======================================================
------------------------------------------------------
======================================================
Waterfall --> Iterative
DevOps has strong roots in Agile but is not just agile
Lean, 7 principles
1 Eliminate Waste
2 Amplify Learning
3 Decide as late as possible
4 Decide as fast as possible
5 Empower the team
6 Build in integrity
7 See the whole picture
Waste
Muda: work that absorbs resources but adds no value
Muri: Unreasonable work imposed on workers and machines
Mura: Work coming in unevenly instead of a constant or regular flow
The 7 Wastes Of Software
1 Partially work done
2 Extar features
3 Relearning
4 Handoffs
5 Delays
6 Task Switching
7 Defects
Build Measure Learn
-----------------------
Build minimum viable product
Measure the outcome and internal metrics
Learn about you problem and your solution
Repeat, go deep where it's needed
Concept to cash: from idea to it's realization, including everything needed to get it to customer
ITSM (Information Technology Service Management)
ITIL (Information Technology Infrastructure Library)
1 Service Strategy
2 Service Design
3 Service Transition
4 Service Operation
ITIL 2007... 2000 pages long... The Visible OPS handbook (only 100 pages)
Use ITIL but make it lean
======================================================
------------------------------------------------------
======================================================
Infrastructure as code
Everything programmatically, no UI, same code runs on test, stage, UAT and production
Servers are cattle not pets.. deploy in mass, not handcrafted
Provisioning: Making a server ready for operation..including OS, network connectivity and system services
Deployment: Automatically deploying and upgrading applications on a server
Orchestration: Performing coordinated operations across multiple systems
Configuration Management: Management of change control after initial provision, maintaining and upgrading apps and app dependencies
Imperative/procedural: Commands necessary to produce a desired state are defined and executed
Declaritive/Functional: A desired state is defined, relying on the tool to configure a system to match that state
Idempotent: The ability to execute repeatedly, resulting in the same outcome
Self Service: The ability for an end user to initiate a process without having to go through other people
Canary (staged) Deployment Pattern
Upgrade 1 server, see how it works before upgrading the rest
Blue Green deployment
Two environment, code is pushed to one, then environments are swapped, variant like cluster immune system deployment
Containers making things easier to deploy, golden image is coming back now since you can just packe it up as a container.
Terraform.. used so that you can use setting across Azure and AWS
---------------------------------------------------------------
Continuous Deploy
Continuous Delivery
Continuous Integration
Time to market goes down... high performing IT orgs can deploy on demand, some 10 or 100 times a day
“Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place.” quote by W. Edwards Deming
With Continuous Delivery you can much easier see what change caused a degradation of an application, with a quarterly release there are so many changes, it is difficult to find out what caused the degradation
The goal of continuous integration is that working software is in a working state all the time quote by Jez Humble
6 practices
Builds should pass the coffee test (should be less than 5 minutes)
Commit really small bits
Don't leave the build broken
Use a trunk based development flow
No crappy tests, fix your tests
The build should return a status, a log and an artifact
Only build artifacts once..immutable, should not change... this creates trust.. underlying bits did not change
Stop deploy if a step fails, notify teams
Test Unit testing
Code hygiene linters, formatters, banned functions check
Integration testing
TDD/BDD/ATDD
TDD
State desired outcome as test
write code to pass test
Repeat
BDD
Work with stakeholders
Describe business functionality
Tests are based on natutal language descriptions
ATDD (acceptance test driven development)
End user perpective
Use case automated testing
testing is continous during development
3 ways to get around slow tests... so you don't violate your coffee rule
use nonblocking tests
use time scheduled tests
use monitoring
MTTR
Mean Time To Recovery
Cascading failure pattern: Cascading failure is number one threat to stability, one layer can choke out other layers
Circuit breaker pattern: If app see an issue with one specific set of calls redirect to other part.. minimizes outage risk
The twelve-factor methodology
https://12factor.net/
I. Codebase
One codebase tracked in revision control, many deploys
II. Dependencies
Explicitly declare and isolate dependencies
III. Config
Store config in the environment
IV. Backing services
Treat backing services as attached resources
V. Build, release, run
Strictly separate build and run stages
VI. Processes
Execute the app as one or more stateless processes
VII. Port binding
Export services via port binding
VIII. Concurrency
Scale out via the process model
IX. Disposability
Maximize robustness with fast startup and graceful shutdown
X. Dev/prod parity
Keep development, staging, and production as similar as possible
XI. Logs
Treat logs as event streams
XII. Admin processes
Run admin/management tasks as one-off processes
If it hurts, do it more often. The best way to avoid failure is to fail constantly
Monitoring
------------------
Service performance/uptime
software component metrics
system metrics
app metrics
performance
security
5 Principles of log data
Do not collect log data if you never plan to use it
Retain log data for as long as it can be used
Log all you can, alert only what you must respond to, define log levels
Don't make logging more available than production stack
Make log changes as needed
Future
Containers and Serverless (serverless might only run 1 invocation)
Security
100:10:1 (Dev:Ops:Sec)
100 developers, 10 Ops, 1 Security person
The Rugged Manifesto: https://ruggedsoftware.org/
I am rugged and, more importantly, my code is rugged.
I recognize that software has become a foundation of our modern world.
I recognize the awesome responsibility that comes with this foundational role.
I recognize that my code will be used in ways I cannot anticipate, in ways it was not designed, and for longer than it was ever intended.
I recognize that my code will be attacked by talented and persistent adversaries who threaten our physical, economic, and national security.
I recognize these things - and I choose to be rugged.
I am rugged because I refuse to be a source of vulnerability or weakness.
I am rugged because I assure my code will support its mission.
I am rugged because my code can face these challenges and persist in spite of them.
I am rugged, not because it is easy, but because it is necessary and I am up for the challenge.
