Musings, Rants and Ponderings Of A DB Architect: January 2019

Sunday, January 27, 2019

TWID Jan 27th, Salah, SSMS emoji, franz josef, mourning stamps,american gods season 2, PostgreSQL

This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these

This Week I Learned

Continued with the Getting Started with Docker Swarm Mode Pluralsight course
Finished A History of Japan by R.H.P. And J. G. Caiger Mason

Did you know, you can use emojis in output from SSMS?

More info, including the code to run can be found here: Did you know, you can use emojis in output from SSMS?

This Week I Tweeted

Why are glasses so expensive? The eyewear industry prefers to keep that blurry

It’s a question I get asked frequently, most recently by a colleague who was shocked to find that his new pair of prescription eyeglasses cost about $800. Why are these things so damn expensive? The answer: Because no one is doing anything to prevent a near-monopolistic, $100-billion industry from shamelessly abusing its market power. Prescription eyewear represents perhaps the single biggest mass-market consumer ripoff to be found. The stats tell the whole story.

Totally agree

The Internals of PostgreSQL for database administrators and system developers

In this document, the internals of PostgreSQL for database administrators and system developers are described.

PostgreSQL is an open source multi-purpose relational database system which is widely used throughout the world. It is one huge system with the integrated subsystems, each of which has a particular complex feature and works with each other cooperatively. Although understanding of the internal mechanism is crucial for both administration and integration using PostgreSQL, its hugeness and complexity prevent it. The main purposes of this document are to explain how each subsystem works, and to provide the whole picture of PostgreSQL.

This document is based on the second part of the book I wrote in Japanese in 2012 (ISBN-13: 978-4774153926) which is composed of seven parts, and covers version 11 and earlier.
Contents

Chapter 1. Database Cluster, Databases and Tables
Chapter 2. Process and Memory Architecture
Chapter 3. Query Processing
Chapter 4. Foreign Data Wrappers (FDW) and Parallel Query
Chapter 5. Concurrency Control
Chapter 6. VACUUM Processing
Chapter 7. Heap Only Tuple (HOT) and Index-Only Scans
Chapter 8. Buffer Manager
Chapter 9. Write Ahead Logging (WAL)
Chapter 10. Base Backup and Point-In-Time Recovery (PITR)
Chapter 11. Streaming Replication

What are you wating for.. start reading.....

Since we are talking about PostgreSQL.....

Microsoft acquires Citus Data, re-affirming its commitment to Open Source and accelerating Azure PostgreSQL performance and scale

I am thrilled to announce that we have acquired Citus Data, a leader in the PostgreSQL community. Citus is an innovative open source extension to PostgreSQL that transforms PostgreSQL into a distributed database, dramatically increasing performance and scale for application developers. Because Citus is an extension to open source PostgreSQL, it gives enterprises the performance advantages of a horizontally scalable database while staying current with all the latest innovations in PostgreSQL. Citus is available as a fully-managed database as a service, as enterprise software, and as a free open source download.

Since the launch of Microsoft’s fully managed community-based database service for PostgreSQL in March 2018, its adoption has surged. Earlier this month, PostgreSQL was named DBMS of the Year by DB-Engines, for the second year in a row. The acquisition of Citus Data builds on Azure’s open source commitment and enables us to provide the massive scalability and performance our customers demand as their workloads grow.

Together, Microsoft and Citus Data will further unlock the power of data, enabling customers to scale complex multi-tenant SaaS applications and accelerate the time to insight with real-time analytics over billions of rows, all with the familiar PostgreSQL tools developers know and love.

That's pretty cool..hopefully they will leave the team alone and not do a FoxPro type move..... But since Ballmer is not in charge anymore, I have faith

Massive mortgage and loan data leak gets worse as original documents also exposed

The bucket contained 21 files containing 23,000 pages of PDF documents stitched together — or about 1.3 gigabytes in size. Diachenko said that portions of the data in the exposed Elasticsearch database on Wednesday matched data found in the Amazon S3 bucket, confirming that some or all of the data is the same as what was previously discovered. Like in Wednesday’s report, the server contained documents from banks and financial institutions across the U.S., including loans and mortgage agreements. We also found documents from the U.S. Department of Housing and Urban Development, as well as W-2 tax forms, loan repayment schedules and other sensitive financial information.

This is really, really messed up

Google Search Operators: The Complete List (42 Advanced Operators)

Did you know that Google is constantly killing useful operators?
That’s why most existing lists of Google search operators are outdated and inaccurate.
For this post, I personally tested EVERY search operator I could find.
Here is a complete list of all working, non‐working, and “hit and miss” Google advanced search operators as of 2018.

All you need to step up your Google Fu

Some cool stuff you might enjoy

Austro-Hungarian emperor Franz Josef defaced

See more info here: The revenge of Yugoslavia on Bosnian stamps of 1906

If you are interested in philately, see also, why do some of these stamps have black perforations?

Took this pic of Liverpool player Mo Salah mural near time square

American Gods | Season 2 Official Trailer

Can't wait to watch that, have watched season I and have also read the book last year

Sunday, January 20, 2019

TWID Jan 20th 2019.. ... Crypts of Winterfell, Brexit, re:MARS, remars, myths, trace flags, agile bs

This Week I Learned

Continued with the Getting Started with Docker Swarm Mode Pluralsight course
Finished D'Aulaires' Book of Greek Myths by Ingri d'Aulaire, Edgar Parin d'Aulaire

This Week I Tweeted

MSSQL Tiger Team: Let’s talk about trace flags

So why do we even have trace flags to begin with?
Some trace flags are used to enable enhanced debugging features such as additional logging, memory dumps etc. and are used only when you are working with Microsoft Support to provide additional data for troubleshooting. These trace flags are not ones you want to leave turned on in a production system as they may have a negative impact on your workload. An example of one of these flags would be TF 2551 which is used to trigger a filtered memory dump whenever there is an exception or assertion in the SQL Server process. These trace flags are only used for a short period of time and typically only at the recommendation of Microsoft Support, so they will likely always be around.

Other trace flags are used to alter the behavior of the server in other ways, such as to turn a feature ON or OFF or change the way the database engine manages resources. One example of this type of trace flag is 7752 which was introduced as a knob for something that is default behavior in Azure SQL DB. In a SQL Server (on-prem or IaaS) database that is undergoing recovery and has Query Store (QDS) enabled, user queries will be blocked until all the data required for QDS to start is loaded. This ensures QDS doesn't miss any queries that are executed in that database. In some cases, this can take a long time to complete and you'll see sessions with a wait type of QDS_LOADDB until the QDS becomes available. Turning on TF 7752 makes this process asynchronous so that user queries can proceed while the QDS starts. It's not something we want to make the default behavior in general because it means that some query executions won't be captured by QDS, but it's a tradeoff you might be willing to make in order to reduce the time it takes for a database to become available upon restart or failover. This is the sort of trace flag that we are trying to incorporate into the product in some other way, such as to provide a database-scoped configuration. Moving forward, we hope not to introduce any new trace flags like this, and over time the number of flags of this type should approach zero.

Make sure to read the rest of the post and read the links to the knowledge base articles as well

Broadcast group Discovery shifts jobs from London to Amsterdam

Discovery has become the latest international broadcast group to move European operations to the Netherlands following Britain’s decision to pull out of the EU. Discovery has had a Dutch base since 1989 but has now applied for EU licences for its paid channel portfolio through the Netherlands.

This decision ensures continuity of our services for the view across Europe,’ the company said in a statement.

I have seen a lot of these job shifts over the last year because of Brexit, even the company I work for has moved people to Amsterdam from London. This will be very costly for London.

Bullet Journal Progress Walls and Progress Grids

The one thing I like to have in my Bullet Journal is a progress wall or progress grid
What a progress wall can do for you is to visually let you know how long you are from your goal. You can make it look neat and pretty or as in my case a little messy

Detecting Agile BS(from the US Department Of Defense) [pdf]

Agile is a buzzword of software development, and so all DoD software development projects
are, almost by default, now declared to be “agile.” The purpose of this document is to provide
guidance to DoD program executives and acquisition professionals on how to detect software
projects that are really using agile development versus those that are simply waterfall or spiral
development in agile clothing (“agile-scrum-fall”).

Pretty cool from the department of Defense

MongoDB removed from RHEL 8 beta due to license

Note that the NoSQL MongoDB database server is not included in RHEL 8.0 Beta because it uses the Server Side Public License (SSPL).

Didn't AWS roll out their own version as well?

Lego collecting delivers huge and uncorrelated market returns

A lot of fancy things can be built with Lego sets nowadays. Such as a diversifying portfolio that loads on the Fama-French size factor. Collecting Lego -- yes, the plastic toys made of interlocking bricks that become cars and castles and robots -- returned more than large stocks, bonds and gold in the three decades ending in 2015, says a study by Victoria Dobrynskaya, an assistant professor at Russia’s Higher School of Economics. Aspects of the performance even align with returns sought by owners of smart-beta ETFs.

Man, I wish I still had my space sets from the 70s intact

Some cool stuff you might enjoy

Apple Open-Sources FoundationDB Record Layer

Today, we are excited to announce the open source release of the FoundationDB Record Layer!

From its inception, FoundationDB was designed as a highly scalable key-value store with a simple API. Layers extend the functionality of the database by adding features and data models to allow new storage access patterns. Today we are releasing the FoundationDB Record Layer, which provides relational database semantics on top of FoundationDB. This layer features schema management, indexing facilities, and a rich set of query capabilities. The Record Layer is used in production at Apple to support applications and services for hundreds of millions of users.

Supports ACID transactions

re:MARS, a new AI event for machine learning, automation, robotics, and space

Machine Learning (ML) and Artificial Intelligence (AI) are behind almost everything we do at Amazon. Some of this work is highly visible, such as autonomous Prime Air delivery drones, eliminating checkout lines at Amazon Go, and making everyday life more convenient for customers with Alexa. But much of what we do with AI and ML happens beneath the surface – from the speed in which we deliver packages, to the broad selection and low prices we’re able to offer customers, to automatic extraction of characters and places from books and videos with X-Ray. This is in addition to the unrivaled breadth and depth of AI and ML services that AWS offers businesses of all sizes. And it’s still day one for AI at Amazon.

Today we are excited to announce we are bringing together some of the brightest leaders across science, academia, and business to explore innovation, scientific advancements, and practical applications of AI and ML. re:MARS, Amazon’s new AI event focused on Machine learning, Automation, Robotics, and Space, will take place June 4-7, 2019 at the ARIA Resort & Casino in Las Vegas. Business leaders and technical builders will learn, share, and further imagine how these four fields of study will shape the future of AI.

See also here: https://remars.amazon.com/

Game of Thrones | Season 8 | Official Tease: Crypts of Winterfell (HBO)

We have all been waiting for this... but where are the books??

Monday, January 14, 2019

Bullet Journal Progress Walls and Progress Grids

I got a Bullet Journal as a gift in December and have been messing around with it a little
The one thing I like to have in my Bullet Journal is a progress wall or progress grid
What a progress wall can do for you is to visually let you know how long you are from your goal. You can make it look neat and pretty or as in my case a little messy

I decided to run 750 miles this year and also do 15,000 push ups. This means I will do about 100 push ups every other day and run about 4 miles 4 times a week

What I did for the 17,500 push ups is divide it up by 100 so that I need a wall with 175 bricks

For the 750 mile run, I divided it by 2.5 miles so that I ended up with 300 bricks

Once you are done with fill in the brick in your favorite color, I chose the color red

Here is what my messy walls look like

Once you finish a row, put a date next to it, this way you can quickly see when you completed that section

If you prefer a neater version, you can of course do a grid, here is what the progress grid I created for my SQL Server blog posts looks like

My goal is to write 100 SQL Server Posts in 2019, here is what the grid looks like right now, so far I created 4 post, I have colored those brick in red

You can use a progress wall for all kind of things... maybe you are saving for a vacation and putting $20 away each week... divide the amount you need by 20 and you have how many bricks you need in your wall.

A progress wall is a fun and creative way to keep track of how far you are from your goal

Let me know in a comment what you would use a progress wall for to track progress

Sunday, January 13, 2019

TWID Jan 13th 2019.. Rules of 3. hacked photosynthesis. Containers Killed The Virtual Machine

This Week I Learned

Finished the Getting Started with Docker Pluralsight course

Rules of 3... for survival
3 Minutes for air
3 Days for water
3 Weeks for food

This Week I Tweeted

Scientists Have 'Hacked Photosynthesis' In Search Of More Productive Crops

"This is a very important finding," she says. "It's really the first major breakthrough showing that one can indeed engineer photosynthesis and achieve a major increase in crop productivity."

It will be many years, though, before any farmers plant crops with this new version of photosynthesis. Researchers will have to find out whether it means that a food crop like soybeans actually produces more beans — or just more stalks and leaves.

Then they'll need to convince government regulators and consumers that the crops are safe to grow and eat.

Giant CES fair in Vegas shows US remains too big to ignore for Chinese tech firms even as trade war swirls

Times are tough in 2019 thanks to the US-China trade war and an escalating war of words between Washington and Beijing over tech leadership

Chinese companies at CES all agreed though that while the trade war has adversely impacted their business in the US, it remains a very important market

“We are definitely affected by the tariffs, in fact one of our big US customers is moving their manufacturing operations outside of China to Vietnam to avoid an increase in the cost of doing business,” said Yuki, a saleswoman from Dongguan-based Ruiheng Electronic Co. Ltd., which manufactures power adaptors and circuit boards.

Costco Sells Out of 27-Pound ‘Storage Bucket’ Mac and Cheese With 20-Year Shelf Life

Bad news for those looking to stock up on Costco’s food hit, the 27-pound bucket of macaroni and cheese with 20 years of shelf life: it’s now sold out on the warehouse’s website after going viral on social media.

Who buys this stuff?

Containers Killed The Virtual Machine Star

We predict new enterprise application development will pass a tipping point in 2019 and shift away from legacy virtual machines (VMs) and strongly toward containers and Kubernetes container orchestration.

To be precise, we predict that:

The future is multi-cloud, and multi-cloud means Docker containers with Kubernetes orchestration. Every public cloud has its own APIs, and in that sense, they are all new versions of proprietary mainframes.
“Lift-and-shift” of VMware virtual machines will be more expensive than customers realize. Paying no money up front, in this case to refactor and port applications to Kubernetes, typically means paying more during operations.
Java is not dead. It may play an important middle ground between lift-and-shift and the expense of completely refactoring applications for cloud-native environments. Java may be a light touch version of “move-and-improve.”
It seems likely that Lenovo will take a look at acquiring SUSE, with Supermicro perhaps also in the mix.

Docker/Kubernetes and other container technologies is all the rage now...especially with DevOps

Print.. the disruptor of batch deletes in SQL

A SQL Server post showing you how the print statement might mess up your batch delete statement

Dozens of government websites rendered insecure or inaccessible due to 80+ expired TLS certificates that haven't been renewed due to the US government shutdown

As the U.S. federal shutdown continues, dozens of U.S. government websites have been rendered either insecure or inaccessible due to expired transport layer security (TLS) certificates that have not been renewed.

In fact, .gov websites are using more than 80 TLS certificates that have expired, according to a new Thursday report by Netcraft. That’s because funding for renewals has been paused. That opens the impacted sites to an array of cyber-attacks; most notably, man-in the-middle attacks, which allow bad actors to intercept exchanges between a user and a web application—either to eavesdrop or to impersonate the website and steal any data that the user may input.

That's not good... shouldn't these people be on the essential employee list?