Outsider Data Science

Putting what’s in there, out there. With R!

Note

This blog is for showing my experiments in R coding. If you are interested in my musings on investments, my LinkedIn posts are the place to find them. You can also visit artsteinmetz.com for more info.

Predicting Water Quality in New York Harbor

Using Tidymodels and data from the Billion Oyster Project

The Truth About Tidy Wrappers

Are Tidyverse wrappers around powerful database engines really worth it?

Kakhovka Dam Disaster

Animate Flood Inundation Along the Dnipro River

Animating a Heatmap with Rayshader

The best part of any analysis are the visualizations. I couldn’t resist taking the results of our earlier sentiment analysis and turning it into some appealing eye candy.

Sentiment Analysis Using Google Translate (Pt. 4 - A Painful Sight)

twitter

tidytext

text mining

Many of us in the U.S. have been surprised by the indifference many in less developed countries have shown about the Ukraine war. There has been much talk lately about how…

Sentiment Analysis Using Google Translate (Pt. 3 - Machine Learning)

tidytext

twitter

machine learning

xgboost

In Part 2 of this series we learned that summing the valence measurement of the words in the Afrisenti data set did not agree well with the sentiments already provided.…

Sentiment Analysis Using Google Translate (Pt. 2 - Word Valence)

tidytext

twitter

Sentiment analysis is a common task in natural language processing. Many of the tools available for this are calibrated to the English language. Can Google Translate let us…

Sentiment Analysis Using Google Translate (Pt. 1 - Translation)

tidyverse

google translate

twitter

Can an English speaker use Google Cloud Translate to conduct experiments with natural language processing? Let’s fund out. Like all my data science posts, I find interesting…

Switching to Quarto from Blogdown

quarto

blogging

It all started when I decided to change up my Hugo theme. Up until that point I was happily using the Blogdown add-in for RStudio to initiate new blog posts. At some point…

Covid Cases vs. Deaths

ggplot2

tidymodels

tidyverse

COVID

Estimate the average lag between a positive COVID-19 case and a death.

What Do The Ramones Want?

ggplot2

tidytext

music

Recently I saw a tweet that shared this hilarious poster of Ramones “wants”. Very cool, but how accurate is it? I asked the graphic designer and he says he took some…

State Taxes: It’s not just about Income

tax

plotly

Much of the discussion around tax burdens focuses on income taxes but, at the state level, that leaves out two other big sources of tax liability, sales and property taxes.…

Gender Diversity in R and Python Package Contributors

github

gender

Over the last few years I have really enjoyed becoming part of the R community. One of the best things about the community is the welcoming, inclusive and supportive nature…

Why I migrated from Excel to R

R

Excel

I’ve been a spreadsheet power user from the days of Visicalc for the Apple II. I migrated to Lotus 1-2-3, to Borland Quattro and finally to Excel. With Excel, I’ve…

Solving the Letterboxed Puzzle in the New York Times

puzzle

recursion

What is the difference between “computer programming” and “data science?” To someone not invovled in either they look much the same. Most data scientists are also coders…

Arthur Steinmetz

Where Are The Libertarians?

politics

ggplot2

At the risk of losing half my readers in the first paragraph, I’ll share my political views. Generally, I believe in “free people and free markets.” That makes me a small-L…

Rick and Morty Palettes

clustering

rick and morty

palettes

This was just a fun morning exercise. Let’s mix multiple images to make a palette of their principal colors using k-means. We’ll also use the totally awesome list-columns…

Is Free Pre-K in NYC Favoring the Rich?

web scraping

education

maps

A hallmark of mayoral administration of NYC Mayor Bill DeBlasio has been free pre-K for all New York families. When the program was initially rolled out there were complaint…

New Winter Sports for New Countries

web scraping

sports

Norway is a tiny country that punches way above its weight in the Winter Olympic medal count. We are not surprised as those folks are practically born on skis. At the same…

Live Fast, Die Young, Stay Pretty?

music

web scraping

health

gapminder

Live fast, die young, stay pretty? That’s the stereotype for rockers, or it was. We only need to look at Keith Richards, over 70 and going strong, to find a major…

Plumbing the Depths of My Soul (in Facebook)

text mining

facebook

First post! Let’s start out nice and easy. No big data machine learning or heavy stats. This post will merely explore the depths of my soul through a meta-analysis of every…