Monday, December 3, 2018

Using PostgreSQL's Interval to mimic SQL Server's DATEADD function

I wanted to do some date calculations in PostgreSQL and was doing some research on if something like the DATEDIFF function that exists in SQL Server is available in PostgreSQL. These notes are mostly for me so I can refer back to them..but maybe they are useful for someone else as well

In SQL Server to add dates, minutes and other fractions of a date to a date, you can use the DATEADD function

Here are some quick examples, if you want to run this in SQL Server, create this table with one row first

CREATE  TABLE test(SomeDate date);


And here is a simple DATEADD query, that adds 1 or 2 days to a date by using both the datepart and the abbreviated datepart

SELECT SomeDate,
  DATEADD(dd,1,SomeDate) as Interval1dd,
  DATEADD(dd,2,SomeDate) as Interval2dd,
  DATEADD(day,1,SomeDate) as Interval1Day,
  DATEADD(day,2,SomeDate) as Interval2Day
FROM test

That will give us the following output

SomeDate Interval1dd Interval2dd Interval1Day Interval2Day
2012-01-01 2012-01-02 2012-01-03 2012-01-02 2012-01-03

If you want to go negative, all you have to do is place a minus sign in front of the number

SELECT SomeDate,
  DATEADD(dd,-1,SomeDate) as Interval1dd,
  DATEADD(dd,-2,SomeDate) as Interval2dd,
  DATEADD(day,-1,SomeDate) as Interval1Day,
  DATEADD(day,-2,SomeDate) as Interval2Day
FROM test

Here is the output of that query

SomeDate Interval1dd Interval2dd Interval1Day Interval2Day
2012-01-01 2011-12-31 2011-12-30 2011-12-31 2011-12-30

Here are all the valid datepart arguments in SQL Server

datepart Abbreviations
year yy, yyyy
quarter qq, q
month mm, m
dayofyear dy, y
day dd, d
week wk, ww
weekday dw, w
hour hh
minute mi, n
second ss, s
millisecond ms
microsecond mcs
nanosecond ns

Now let's take a look at PostgreSQL

In PostgreSQL, there is no DATEPART function, but you can use interval literals to accomplish something that behaves the same

Let's do something similar like we did with the SQL Server queries, first create this temp table with one row

CREATE Temp TABLE test(SomeDate date);

VALUES( to_date('20120101','YYYYMMDD'));

Now let's run this query

SELECT SomeDate,(SomeDate + 1 * INTERVAL '1 Day' ) as Interval1Day,
  (SomeDate + 1 * INTERVAL '1 D')  as IntervalD,
  (SomeDate + 2 * INTERVAL '1 Day' ) as Interval2Times1Day,
  (SomeDate + 1 * INTERVAL '2 Day' ) as Interval2Days,
  (SomeDate + 2 * INTERVAL '2 Days' ) as Interval2Times2Days,
  (SomeDate +  INTERVAL '1 Day' )    as IntervalD
FROM test

Here is the output

"2012-01-01";"2012-01-02 00:00:00";"2012-01-02 00:00:00";"2012-01-03 00:00:00";"2012-01-03 00:00:00";"2012-01-05 00:00:00";"2012-01-02 00:00:00"

When you copy from pgAdmin , you don't get the column aliases, so below is a screenshot of what it looks like(Click on the image for a bigger sized image)

As you can see there are two parts where you can supply a number

I think I prefer the top one from the query below, since it resembles the DATEPART function more

(SomeDate + 2 * INTERVAL '1 Day' ) as Interval2Times1Day,
(SomeDate + 1 * INTERVAL '2 Day' ) as Interval2Days,

But as you saw,it was possible to add 4 days by using a one in both places like shown below

(SomeDate + 2 * INTERVAL '2 Days' ) as Interval2Times2Days,

And of course if you want, you can just use the number inside the string like in this example below

(SomeDate +  INTERVAL '1 Day' )    as IntervalD

It's up to you, but I don't like changing numbers inside a string

To do negative numbers, you just change the positive number to a negative number, here is the same query from before but now with negative numbers

SELECT SomeDate,(SomeDate + -1 * INTERVAL '1 Day' ) as Interval1Day,
  (SomeDate + -1 * INTERVAL '1 D')    as IntervalD,
  (SomeDate + -2 * INTERVAL '1 Day' ) as Interval2Times1Day,
  (SomeDate + -1 * INTERVAL '2 Day' ) as Interval2Days,
  (SomeDate + -2 * INTERVAL '-2 Days')as Interval2Times2Days,
  (SomeDate +  INTERVAL '-1 Day' )    as IntervalD
FROM test

Here is the output
"2012-01-01";"2011-12-31 00:00:00";"2011-12-31 00:00:00";"2011-12-30 00:00:00";"2011-12-30 00:00:00";"2012-01-05 00:00:00";"2011-12-31 00:00:00"

As you can see that all works as expected, did you notice that the we get +4 when we do -2 * -2?

Here is the output also from pgAdmin so that you can see the column aliases

Besides using days, you can also use these parts of a date

Abbreviation Meaning
Y Years
M Months (in the date part)
W Weeks
D Days
H Hours
M Minutes (in the time part)
S Seconds

Let's take a look by using some of these in a query

SELECT SomeDate,(SomeDate + 1 * INTERVAL '1 Day' ) as Interval1Day,
  (SomeDate + 1 * INTERVAL '1 Week' ) as Interval1Week,
  (SomeDate + 1 * INTERVAL '1 Month' ) as Interval1Month,
  (SomeDate + 1 * INTERVAL '1 Year' ) as Interval1Year
FROM test
SELECT SomeDate,(SomeDate + 1 * INTERVAL '1 D' ) as Interval1Day,
  (SomeDate + 1 * INTERVAL '1 W' ) as Interval1Week,
  (SomeDate + 1 * INTERVAL '1 M' ) as Interval1Month,
  (SomeDate + 1 * INTERVAL '1 Y' ) as Interval1Year
FROM test

Here is the output

"2012-01-01";"2012-01-02 00:00:00";"2012-01-08 00:00:00";"2012-02-01 00:00:00";"2013-01-01 00:00:00"
"2012-01-01";"2012-01-02 00:00:00";"2012-01-08 00:00:00";"2012-01-01 00:01:00";"2013-01-01 00:00:00"

Here is the output also from pgAdmin so that you can see the column aliases

As you can see when using M it used minute not month. I would recommend to always use the full name and not the abbreviated part so as not to create confusion

That's all for this post

Sunday, November 25, 2018

Easy running totals with windowing functions in PostgreSQL

Back in the pre windowing function days, if you wanted to do a running count, you either had to run a subquery or you could use a variable. This was slow because for each row the query that did the sum would be executed. With windowing functions in PostgreSQL, this is now running much faster. 

Let's take a look, first create the following table

CREATE Temp TABLE test(Id int,SomeDate date, Charge decimal(20,10));

insert into test
values( 1,to_date('20120101','YYYYMMDD'),1000);
insert into test
values( 1,to_date('20120401','YYYYMMDD'),200);
insert into test
values( 1,to_date('20120501','YYYYMMDD'),300);
insert into test
values( 1,to_date('20120601','YYYYMMDD'),600);
insert into test
values( 2,to_date('20120101','YYYYMMDD'),100);
insert into test
values( 2,to_date('20120101','YYYYMMDD'),500);
insert into test
values( 2,to_date('20120101','YYYYMMDD'),-800);
insert into test
values( 3,to_date('20120101','YYYYMMDD'),100);

let's check that data we just inserted into the temporary table


The output looks like this

Id SomeDate Charge
1 2012-01-01 1000.0000000000
1 2012-04-01 200.0000000000
1 2012-05-01 300.0000000000
1 2012-06-01 600.0000000000
2 2012-01-01 100.0000000000
2 2013-01-01 500.0000000000
2 2014-01-01 -800.0000000000
3 2012-01-01 100.0000000000

What we want is the following

id StartDate Enddate         Charge         RunningTotal
1 2012-01-01 2012-03-31 1000.0000000000 1000.0000000000
1 2012-04-01 2012-04-30 200.0000000000 1200.0000000000
1 2012-05-01 2012-05-31 300.0000000000 1500.0000000000
1 2012-06-01 9999-12-31 600.0000000000 2100.0000000000
2 2012-01-01 2012-12-31 100.0000000000 100.0000000000
2 2013-01-01 2013-12-31 500.0000000000 600.0000000000
2 2014-01-01 9999-12-31 -800.0000000000 -200.0000000000
3 2012-01-01 9999-12-31 100.0000000000 100.0000000000

For each row, we want to have the date that the row starts on and also the date when it ends, we also want a running total as well. If there is no row after the current row for that id, we want the end date to be 9999-12-31.

So we will use a couple of functions. The first one is LEAD, LEAD accesses data from a subsequent row in the same result set without the use of a self-join. So the LEAD part looks like this

LEAD((SomeDate + -1 * INTERVAL '1 day' ),1,'99991231') OVER (PARTITION BY id ORDER BY SomeDate) as Enddate,

What we are doing is subtracting 1 from the date in the subsequent row (SomeDate + -1 * INTERVAL '1 day' )
We are using 1 as the offset since we want to apply this to the next row. Finally if there is no subsequent row, we want to use the date 9999-12-31 instead of NULL

To do the running count, we will do the following

          AS RunningTotal

What this means in English is for each id ordered by date, sum up the charge values for the rows between the preceding rows and the current row. Here is what all that stuff means.

Specifies the rows that make up the range to use as implied by

Specifies that the window starts at the first row of the partition. UNBOUNDED PRECEDING can only be specified as window starting point.

Specifies that the window starts or ends at the current row when used with ROWS or the current value when used with RANGE.
CURRENT ROW can be specified as both a starting and ending point.

And here is the query

SELECT id, someDate as StartDate,
LEAD((SomeDate + -1 * INTERVAL '1 day' ),1,'99991231')
 OVER (PARTITION BY id ORDER BY SomeDate) as Enddate,
          AS RunningTotal
  FROM test
  ORDER BY id, SomeDate

And running that query, gives us the running count as well as the end dates 

id StartDate Enddate         Charge         RunningTotal
1 2012-01-01 2012-03-31 1000.0000000000 1000.0000000000
1 2012-04-01 2012-04-30 200.0000000000 1200.0000000000
1 2012-05-01 2012-05-31 300.0000000000 1500.0000000000
1 2012-06-01 9999-12-31 600.0000000000 2100.0000000000
2 2012-01-01 2011-12-31 100.0000000000 100.0000000000
2 2012-01-01 2011-13-31 500.0000000000 600.0000000000
2 2012-01-01 9999-12-31 -800.0000000000 -200.0000000000
3 2012-01-01 9999-12-31 100.0000000000 100.0000000000

Here is what it looks like if you execute the query in PGAdmin

If you don't want the last row to have the end date filled in, just omit the default value in the LEAD function. Instead of

LEAD((SomeDate + -1 * INTERVAL '1 day' ),1,'99991231')

Make it

LEAD((SomeDate + -1 * INTERVAL '1 day' ),1)

Here is the whole query again

SELECT id, someDate as StartDate,
LEAD((SomeDate + -1 * INTERVAL '1 day' ),1)
 OVER (PARTITION BY id ORDER BY SomeDate) as Enddate,
          AS RunningTotal
  FROM test
  ORDER BY id, SomeDate
And here is what the output looks like after we made the change
As you can see the rows for an id that doesn't have a row with a date greater than the current date will have a null end date
That is all for this post

Friday, November 23, 2018

Happy Fibonacci day, here is how to generate a Fibonacci sequence in PostgreSQL

Image by Jahobr - Own work, CC0, Link

Since today is Fibonacci day I decided to to a short post about how to do generate a Fibonacci sequence in PostgreSQL. But first let's take a look at what a Fibonacci sequence actually is.

In mathematics, the Fibonacci numbers are the numbers in the following integer sequence, called the Fibonacci sequence, and characterized by the fact that every number after the first two is the sum of the two preceding ones:

 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

Often, especially in modern usage, the sequence is extended by one more initial term:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

November 23 is celebrated as Fibonacci day because when the date is written in the mm/dd format (11/23), the digits in the date form a Fibonacci sequence: 1,1,2,3.

So here is how you can generate a Fibonacci sequence in PostgreSQL, you can do it by using s recursive table expression.  Here is what it looks like if you wanted to generate the Fibonacci sequence to up to a value of 1 million

;WITH RECURSIVE Fibonacci (Prev, Next) as
     SELECT 0, 1
     SELECT Next, Prev + Next
     FROM Fibonacci
     WHERE Next < 1000000
SELECT Prev as Fibonacci
     FROM Fibonacci
     WHERE Prev < 1000000

That will generate a Fibonacci sequence that starts with 0 if you need the Fibonacci sequence to start at 1, all you have to do is replace the 1 to 0 in the first select statement

;WITH RECURSIVE Fibonacci (Prev, Next) as
     SELECT 1, 1
     SELECT Next, Prev + Next
     FROM Fibonacci
     WHERE Next < 1000000
SELECT Prev as Fibonacci
     FROM Fibonacci
     WHERE Prev < 1000000

Here is what it looks like in PGAdmin when you run the query

Happy Fibonacci day!!

Here is pretty much the same post that I created for SQL Server: Happy Fibonacci day, here is how to generate a Fibonacci sequence in SQL

Tuesday, November 20, 2018

This brought back memories

The other day I walked past the building in the picture above and I felt a little sad. The reason I felt sad is because I have great memories about this building. When I first came to the US from the Netherlands I was amazed at how big this store was and how they had so many things on display. In Amsterdam where I arrived from, you didn’t really have a store like this. The story was named J&R, there was a J&R Music World store and a J&R Computer World store. J&R stands for the founders Joe and Rachelle Friedman.

There were several items I bought at this store

The first camera I ever owned I bought in this place, it was a Minolta but I don’t remember the model. I later sold the camera because a co-worker needed the camera while he went on vacation to Equador. So I only had it for 2 or 3 months.

A couple of years later I replaced that camera with a Canon 50E  I think this Canon camera might have also been known as an Elan. The E in the model name stands for eye-control, this was an enhanced version of the 3-zone eye-controlled autofocus system that was first seen on the EOS 5 camera. This was very high tech back then.

Another item that I have fond memories about was the first MP3 player that I purchased; It was the first MP3 player that was available to buy. It was the Rio PMP-300 portable MP3 player. I believe it was around $200 and you could hold about 10 songs or so since it only came with 32 MB of memory. You could store more songs if you inserted a smartmedia card. But even then you could not store hundreds of songs. I remember encoding the songs in WMA 64k format so I could store even more songs…… lol  1st world problems.

I bought many other things at J&R like camera lenses, filters for the camera, a tripod, RAM to upgrade the PC, I even purchased Plus! lol  Plus had themes and utilities for Windows

The nice thing about J&R was that they had prices unlike 42 street photo or whatever those stores were called where you had to negotiate the price. Also unlike at the Nobody Beats The Wiz store, they did not try to sell you insurance with every item you bought.

These days I guess you would have to go to B&H to get a  similar experience, just be aware that they are not open on Saturdays.

Monday, April 23, 2018

TWID April 23, 2018

This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these

This week my youngest son and I decided to read all the Lord Of the Rings books. Both of us have seen the movies many times and I have read the books back in 2001. We decided to both sit in a room and I read out loud to him. It is kind of hard to read out load... I guess last time I had to read in front of people was back in high school

Finished 1965: The Most Revolutionary Year in Music
This was a fun book, 1965 was the year of protest, drugs and very good music. Highly recommended.
After I was done with the book, I decided to get some of this music mentioned in this book from Amazon

Got these three songs

While My Guitar Gently Weeps (Remastered)
From the album The Beatles (Remastered) [Explicit]
By: The Beatles

The Sound of Silence (Overdubbed Version)
From the album The Best Of Simon & Garfunkel
By: Simon & Garfunkel

Like a Rolling Stone
From the album Bob Dylan's Greatest Hits
By: Bob Dylan

Except for Sound of Silence, I already had the other two songs somewhere but I was too lazy to go searching for it. I watched the movie Watchmen a week or so ago and that is when I heard the song Sound of Silence, now that it was mentioned in the book I had to get it. Surprisingly I don't believe I have ever heard this song before watching the movie.

Got a couple of other songs as well

Damn It Feels Good to Be a Gangsta [Explicit]
From the album Uncut Dope [Explicit]
By: Geto Boys

You can hear this song in the movie Office Space... the best part is when someone is rapping pretending to be the president and it sounds just like Bill Clinton

Shout 2000
From the album The Sickness [Explicit]
By: Disturbed

This is a version of the Tears N' Fears song by the band Disturbed

So Long, And Thanks For All The Fish
From the album Eat The Elephant [Explicit]
By: A Perfect Circle

Was on the bestseller list.. sounded interesting, so I got iy

Paid In Full (Seven Minutes Of Madness - The Coldcut Remix)
From the album Paid In Full (Deluxe Edition)
By: Eric B. & Rakim

When I was a kid in 1987 I listened to this mix with the song Im Nin'Alu by singer Ofra Haza
If you know the song Girl You Know It's True by Milli Vanilli you will recognize the drum beat. This is one of those classic rap song that you wished were still made today

This Week I Learned

Watched parts of the SQL Server 2016 New Features for Developers Pluralsight course

This Week I tweeted

GPUs Mine Astronomical Datasets for Golden Insight Nuggets 

Critical to all of this research were GPU accelerators – specifically the Tesla P100s used in the DGX-1 server from Nvidia – which enabled accelerated training of neural networks. They used the Wolfram Language neural network functionality, built a top of the open-source MXNet framework, that in turn uses the cuDNN library for accelerating the training on Nvidia GPUs. ADAM was deployed as the underlying learning algorithm. The significant horsepower of the Blue Waters system, which is also GPU accelerated, was brought to bear for their modeling data and for solving Einstein’s equations via simulation. The group are also looking into generative models GANs (generative adversarial networks) to further reduce the multi-week time taken (even for Blue Waters) for these specific steps. 

I am still waiting for the day that SQL Server will be able to use GPUs in addition to CPUs

Scientists make new plastic-eating enzyme in fight against pollution

A team of scientists in Britain and at the U.S. Department of Energy say that they have bolstered the ability of an enzyme discovered in Japan to eat the plastic found in soda bottles.

The plastic polyethylene terephthalate (PET) normally lasts for hundreds of years, but a release from the University of Portsmouth and the National Renewable Energy Laboratory says that a new bacteria will be able to chow down and speed up the process to deal with the huge amounts of waste humans make.

This is pretty cool, in the end technology will be used to solve the problem that technology created

Light-Powered Camera The prototype gets 15 frames/second, no external power needed

Light is used for both image sensing and solar power.

This is awesome and scary at the same time... awesome because you can have cameras that you don't need to charge, scary because you know these cameras will be deployed everywhere and will be recording everything people do.

Microsoft built its own custom Linux kernel for its new IoT service

At RSA 2018, Microsoft announced the preview of Microsoft Azure Sphere, a new solution for creating highly-secured, Internet-connected microcontroller (MCU) devices. Azure Sphere includes three components that work together to protect and power devices at the intelligent edge.
  • Azure Sphere certified microcontrollers (MCUs):A new cross-over class of MCUs that combines both real-time and application processors with built-in Microsoft security technology and connectivity. Each chip includes custom silicon security technology from Microsoft, inspired by 15 years of experience and learnings from Xbox, to secure this new class of MCUs and the devices they power.

  • Azure Sphere OS: This OS is purpose-built to offer unequalled security and agility. Unlike the RTOSes common to MCUs today, our defense-in-depth IoT OS offers multiple layers of security. It combines security innovations pioneered in Windows, a security monitor, and a custom Linux kernel to create a highly-secured software environment and a trustworthy platform for new IoT experiences. 

  • Azure Sphere Security Service: A turnkey, cloud service that guards every Azure Sphere device; brokering trust for device-to-device and device-to-cloud communication through certificate-based authentication, detecting emerging security threats across the entire Azure Sphere ecosystem through online failure reporting, and renewing security through software updates. It brings the rigor and scale Microsoft has built over decades protecting our own devices and data in the cloud to MCU powered devices. 

  • These capabilities come together to enable Azure Sphere to meet all 7 properties of a highly secured device – making it a first of its kind solution.

Here is a short video that has also some information

FDA clears first contact lens with light-adaptive technology

The U.S. Food and Drug Administration today cleared the first contact lens to incorporate an additive that automatically darkens the lens when exposed to bright light. The Acuvue Oasys Contact Lenses with Transitions Light Intelligent Technology are soft contact lenses indicated for daily use to correct the vision of people with non-diseased eyes who are nearsighted (myopia) or farsighted (hyperopia). They can be used by people with certain degrees of astigmatism, an abnormal curvature of the eye.

This sounds cool but if it takes 10 seconds for the lenses to go back to normal clear view you might have issues when going into a tunnel when you drive with these lenses in.

Some cool stuff you might enjoy

Glass for geeks: An in-depth tour of Nikon’s Hikari Glass factory

I've been on a lot of factory tours with various camera and lens manufacturers before, but had never had a chance to see how the optical glass was made that goes into the lenses we use every day. So I was really happy to receive an invite from Nikon to tour their Hikari Glass factory in Akita Japan, following the annual CP+ trade show in Yokohama this year.

This was a pretty special tour, as we got to see the whole process, from start to finish, hosted by three of Hikari's top executives. Our hosts were Mr. Tatsuo Ishitoya, President-Director, Mr. Akio Arai, Corporate Vice President and Production General Manager, and Mr. Toshihiko Futami, Director and Management General Manager. Mr. Masaru Kobayashi, Assistant Manager of the Administration Section also accompanied us and contributed to the information we received. Arai-san is the person directly responsible for plant operations, and it was him who personally guided us on our extensive tour. All three executives briefed us before and after the tour itself.

Wow, I never thought about how complex this stuff is. Great article

Monday, April 16, 2018

TWID April 16, 2018

This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these

This Week I Learned

Finished the Understanding Machine Learning with Python Pluralsight course

Understanding Machine Learning with Python

This was a pretty interesting course and it's perfect for a beginner. You don't need to know any AI, you also don't need to know much Python either

This Week I tweeted

Oops... connected to the wrong server

On Tuesday, 13 March 2018 at 12:04 UTC a database query was accidentally run against our production database which truncated all tables.

It took us a day to uncover the root cause of the original database truncation. Using our API logs, and with information from our upstream provider about the IP address the query originated from, we were able to identify a truncate query run during tests using the Database Cleaner gem. The shell the tests ran in unknowingly had a DATABASE_URL environment variable set as our production database. It was an old terminal window in a tmux session that had been used for inspecting production data many days before. The developer returned to this window and executed the test suite with the DATABASE_URL still set.

This is why getting to the production DB should only be enabled on a handful of servers, which few people can access. Deploy to prod from there, test, staging and dev can be done from less restrictive machines. Never should you build both from the same machines!

The sale of electric cars more than doubles in first quarter in the Netherlands

The sale of battery-powered electric cars increased by 136% to 3,945 in the first three months of 2018, motoring organisation RAI Vereniging reported on Monday. The market for electric cars, so far, is miniscule, but the Netherlands is the world’s fifth-largest market for electric cars after China, the US, Japan and Norway, according to the International Energy Agency.
It's still a small number in the grand scheme of things but it is promising. Wondering if the Tesla 3 model will make an impact.

Check if facebook shared your info with Cambridge Analytica

Recently, we shared information about the potential misuse of your Facebook data by apps and websites. We also shared plans for how we're taking action to prevent this from happening in the future.
Check below to see if your information may have been shared with Cambridge Analytica by the app “This Is Your Digital Life.”

It looks like my data wasn't shared.. but then again I don't take silly quizzes. I also don't have facebook installed on my current phone which I got in December, neither have I ever logged in from the PC I am typing from at the moment.

Why Does “=” Mean Assignment? 

A common FP critique of imperative programming goes like this: “How can a = a + 1? That’s like saying 1 = 2. Mutable assignment makes no sense.” This is a notation mismatch: “equals” should mean “equality”, when it really means “assign”. I agree with this criticism and think it’s bad notation. But I also know some languages don’t write a = a + 1, instead writing a := a + 1. Why isn’t that the norm?

The usual answer is “because of C”. But that’s just passing the buck: why does C do it that way? Let’s find out!

I also always though it's because of C  :-)  Now you know why :-)

A massive, 'semi-infinite' trove of rare-earth metals has been found in Japan

Researchers have found hundreds of years' worth of rare-earth materials underneath Japanese waters — enough to supply to the world on a "semi-infinite basis," according to a study published in Nature Publishing Group's Scientific Reports.
Rare-earth metals are crucial in the making of high-tech products such as electric vehicles and batteries, and most of the world has relied on China for almost all of its needs.

Japan started seeking its own rare-earth metals after China held back shipments in 2010 during a dispute over islands both countries claim. These island are the Senkaku Islands, they are also known as the Diaoyu Islands and the Pinnacle Islands

Unusual Homes Around the World

Some of these houses are very unique, strange and in some cases very impressive. The Heliodome one looks really interesting.

Would you want to live in some of these?

See also for a documentary about unusual homes. I like the house that uses the wing of an airplane as its roof

Some cool stuff you might enjoy

DatabaseFlow -- an open source self-hosted SQL client

Database Flow is an open source self-hosted SQL client, GraphQL server, and charting application that works with your database. Visualize schemas, query plans, charts, and results. You can run Database Flow locally for your own use, or install to a shared server for the whole team.

Here are some screenshots of what it looks like

Electricity Map

This shows in real-time where your electricity comes from and how much CO2 was emitted to produce it.

You can see what it looks like here

Some pics I took

Found this Indian Head penny from 1900 in my change. As you can see it's pretty worn out and not worth more than 25 cents probably. But it's still a nice piece of history to have. I think the older US coins were much nicer looking that the current ones. The Morgan dollar coin is probably one of the best looking coins ever made.

Sunday, April 8, 2018

TWID April 08, 2018

This is a post detailing some stuff I did, learned, posted and tweeted this week, I call this TWID (This week in Denis). I am doing this mostly for myself... a kind of an online journal so that I can look back on this later on. Will use the label TWID for these

I skipped a couple of weeks but decided to get back to post these again

Went to Washington DC with the family during spring break. I always wanted to see the cherry blossoms in DC but so far every time I visited Washington DC it was in the middle of the summer. This year I got lucky because it was colder than normal for March so the cherry trees didn't bloom until the first week of April.

Here is a pic I took of the Jefferson Memorial with the cherry blossoms in the foreground.

Jefferson Memorial through Cherry Blossoms

The plan was to visit NPR studios, see the Air and Space museum in Virginia and then to hit a museum or two.

The drive from Princeton to the Air and Space museum was not bad, it took a little under 4 hours. There were a couple of things I wanted to see in the museum, these were

The Space Shuttle
The Enola Gay plane
The Concord
World War II fighter planes

Here is the space shuttle

Space Shuttle

For some reason I thought the space shuttle was bigger, I guess I was comparing the shuttle to a Boeing 747

We took a tour and the tour guide was quite good, he had some interesting stories and facts that he gave us

Enola Gay Plane

In the pic above you see the Enola Gay plane, this is the plane that was used to drop the atomic bombs on Hiroshima and Nagasaki. The tour guy told us that when they were training in the US with this plane, there was only one person that knew what the real mission would be. It was interesting to see how shiny this plane was. I wonder who has to keep these plane polished and how long it takes
Here is also a close up of the cockpit
  Enola Gay Cockpit

The plane below is a Hawker Hurricane IIC

Hawker Hurricane IIC
The Hawker Hurricane IIC plane was the first British monoplane fighter and the first  British  fighter to exceed 483 kilometers (300 miles). Hurricane fighters fought the Luftwaffe and helped win the battle of Britain in the summer of 1940

The plane below is a Heinkel He 219 A Uhu

Heinkel He 219 A Uhu

A Heinkel He 219 A Uhu was armed with up to 8 cannons and guided to its target by radar. The Heinkel He 219 A Uhu (Eagle Owl) was one of the Luftwaffe most formidable night fighters. On the aircraft's first mission, a single Heinkel He 219 A Uhu shot down at least five British bombers

Here are some Luftwaffe planes with swastikas on them Luftwaffe Nazi Planes

Finally here you can see the 'real' Air Jaws :-)

  The real Air Jaws

The Air and Space museum in Virginia is really cool, it is bigger than the one in Washington DC and the entrance is free as well


NPR building with Capitol in the distance

We took a tour of the NPR facility in Washington DC. The tour was really good, it took an hour and they took us to several floors as well as the roof. Unfortunately, we were not allowed to take pictures on ever floor so I can just share a handful of pics.

I was surprised that NPR had a bunch of windows machines... I expected them all to be Macs or even Linux but nope... all Windows machines. Below is a pic of the backup studio, the microphones come from Germany and are quite expensive

Backup Studio at NPR

Sound Diffuser
What you see in the picture below is a sound diffuser.

Sound Diffuser at NPR

Here is how wikipedia explains what a sound diffuser is

Diffusors (or diffusers) are used to treat sound aberrations, such as echoes, in rooms. They are an excellent alternative or complement to sound absorption because they do not remove sound energy, but can be used to effectively reduce distinct echoes and reflections while still leaving a live sounding space. Compared to a reflective surface, which will cause most of the energy to be reflected off at an angle equal to the angle of incidence, a diffusor will cause the sound energy to be radiated in many directions, hence leading to a more diffusive acoustic space. It is also important that a diffusor spreads reflections in time as well as spatially. Diffusors can aid sound diffusion, but this is not why they are used in many cases; they are more often used to remove coloration and echoes.

Diffusors come in many shapes and materials. The birth of modern diffusors was marked by Manfred R. Schroeders' invention of number-theoretic diffusors in the 1970s.

More here:

In the pic below you can see my daughter Catherine holding the Emmy Awards that one of the Tiny Desk hosts won

Catherine holding the NPR Tiny Desk Emmy

Tiny Desk, yes you read that right... tiny not tidy  :-) Most of the stuff you see in this picture is the stuff that performers left after performing. Adele left her water bottle, I believe they KISS guys also left a water bottle.  Pretty cool to see all this stuff

NPR Tiny Desk

In the pic below is the camera David Gilkey was carrying when he was killed in Afghanistan on June 5th 2016. David Gilkey was a staff photographer and video editor for NPR, covering both national and international news

David Gilkey's Camera

This Week I Learned

Learned a bunch of stuff about PostgreSQL by taking a Pluralsight course

This Week I tweeted

Nikon versus Canon: A Story Of Technology Change

Pretty cool post showing you how Canon gained ground on Nikon

The high-stakes battle for the Pentagon’s winner-take-all cloud contract

There is a battle afoot, one you might not have heard about yet, but it involves a high-stakes winner-take-all contract for the Department of Defense’s cloud contract. It could involve billions of dollars and when a humongous sum of money meets a set of powerful tech companies, intrigue can’t be far behind.

The story even has a Star Wars reference with the Pentagon dubbing the project the Joint Enterprise Defense Infrastructure (or JEDI for short). Who says the Pentagon is staid?

The tech names involved include the likes of Amazon,  Google, Microsoft, Oracle and Rean Cloud, LLC.

Based on Trump's tweets last week... I doubt it will go to AWS

Some cool stuff you might enjoy

It's time to head back to RSS? 

THE MODERN WEB contains no shortage of horrors, from ubiquitous ad trackers to all-consuming platforms to YouTube comments, generally. Unfortunately, there's no panacea for what ails this internet we've built. But anyone weary of black-box algorithms controlling what you see online at least has a respite, one that's been there all along but has often gone ignored. Tired of Twitter? Facebook fatigued? It's time to head back to RSS.

Well I never left RSS, after Google Reader was shut down I switched to Feedly and have been using it ever since

Some pics I took

You can see all my Washington DC pics I posted on flickr here: