3 Questions I was Afraid to Ask (and my Tensorflow 2.0 Template)

Stressed girl chewing on pencil
Stressed girl chewing on pencil
Photo by JESHOOTS.COM on Unsplash

For many people, myself included, Q-learning serves as an introduction to the world of reinforcement learning. It gets us neatly accustomed to the core ideas of states, actions and rewards in a way that is intuitive and not bogged down by complicated technical details.

What makes implementing Deep Q-Learning so much more difficult then? While Q-learning took me only a day to go from reading the Wikipedia article to getting something that worked with some OpenAI Gym environments, Deep Q-learning frustrated me for over a week!

Despite the name, Deep Q-learning is not as simple as swapping out a state-action table for a neural network. For me, becoming comfortable with coding Deep Q Networks (DQNs) took three broad steps in…


A retrospective

Image for post
Image for post
Photo by Austin Distel on Unsplash

In some work categorization frameworks, jobs are placed into quadrants depending on whether they are routine or nonroutine, and cognitive or manual. Data science can be a rewarding and stimulating career path due to its nature as a nonroutine cognitive discipline. However, the way that demands and problems change throughout the course of projects can lead to some uncertainty.

As I approach the two year anniversary of my first non-internship position as a data scientist, I think it would be interesting to record and share my experience so far.

In particular, I’ve identified four typical “phases” of projects I’ve worked on, where the nature of problems I’m faced with feel qualitatively different from each other. For each, I describe the nature of tasks and identify the challenges unique to the phase. By doing so, I hope to shed some light on some periodic and predictable elements of an otherwise nonroutine job. …


3 functions from the Tidyverse package that improve working with lists in R

Image for post
Image for post
Photo by Catherine Heath on Unsplash

The purrr package is one of the mainstays of the Tidyverse, right up there with dplyr and the pipe operator in terms of usefulness and universality. One of its main purposes is to align the intent of a loop with its syntax. Instead of involving a bunch of tedious boilerplate when writing a for-loop — which tells the reader little about what the loop does — a purrr map can be read almost like plain English, and fits neatly into groups of piped-together operations.

But this is not all purrr has to offer. For the longest time, I only used map, walk, and their variants. Since then, I’ve discovered the package’s other utilities that make working with lists a breeze. …


How neural networks are democratizing procedural knowledge

Image for post
Image for post
Photo by Franck V. on Unsplash

Are neural networks and deep learning the keys that will finally unlock the path to artificial general intelligence, or are we on the verge of another AI Winter?

If you’re an entrepreneur, then the answer is: it doesn’t matter. Your funding doesn’t depend on grant providers being excited about your research agenda producing a talking robot. It depends on providing real value to real people.

The truth is, even though developments in AI so far have yet to produce machines that are intelligent in a way comparable to our own intelligence, the technologies that have been developed along the way are all incredibly useful and full of potential value. …


Going beyond the defaults to create pretty and reproducible charts

Image for post
Image for post
Photo by Kevin Ku on Unsplash

Polar Bar Charts can be great ways to display categorical information if you want to draw more attention to the categories with higher values. They were used to great effect in the Wall Street Journal’s article on Amazon’s HQ2 Shortlist. But even with the powerful visualization tools at our fingertips, producing charts like that takes a bit of legwork.

I’m here today to provide you with the pieces of geometrical drudgery needed to turn Plotly’s polar bar chart into something visually acceptable. …


On finding gaps in your STEM education and enhancing your lifelong learning

Image for post
Image for post
Photo by imgix on Unsplash

It’s a tragedy whenever someone gets that degree or lands that job, and puts their days of earnest learning behind them. The truth is, we come away from our formal educations with holes in our abilities. Often, these gaps can be filled in a way that enhances how we perceive the world around us, with only a small investment of time and attention.

I believe that one class of learning gaps comes from the mismatch between the mechanical difficulty and the conceptual difficulty of the things we learn in school. …


A tf.keras Functional API code snippet

Image for post
Image for post
Photo by NESA by Makers on Unsplash

It seems that we’ve become so obsessed with building neural networks in 10 lines of code that we’ve forgotten to build neural networks in reusable lines of code.

Neural networks are useful machine learning models, and they’re maturing as a technology. As a consequence of this, many things that used to be done by hand, like determining the dimensions of the weight matrices or implementing backpropagation, are now taken care of by software frameworks. Of course, many of us are too busy demonstrating that we know how to transpose a matrix in NumPy to notice.

Since building neural networks is now easy and the software for doing so is more stable, there should no longer be any excuse for using flat, ad-hoc code. For the most control, you might favor implementing them as a class. For the minimum amount needed for rapid experimentation and prototyping, a function that spits out a fully workable network is enough. …


Base rates, marginal probabilities, sensitivity, and specificity

Image for post
Image for post
Photo by Markus Spiske on Unsplash

When it comes to updating beliefs and making decisions under uncertainty, Bayes’ theorem is just about the best tool available. And yet it is so often relegated to academic textbooks and machine learning applications when it should be bringing us value in our daily lives.

This article is for those of us who are familiar with Bayes’ theorem, but might have trouble remembering which terms go where and don’t immediately think of it when reading headlines like “red wine reduces dementia risk.”

First, I’ll review Bayes’ theorem, talk about why I think it is hard to remember when it really shouldn’t be, and formulate it in terms of sensitivity and specificity (which I’ll define). Next, I’ll discuss a few ways to use the formula to appropriately react to provocative claims that often appear in the news. Finally, I’ll leave you with some caveats about using Bayes’ theorem in an everyday fashion, and other final thoughts. …


Demonstrating meshgrid, vectorize, and other useful tools

Image for post
Image for post
Photo by Christian Fregnan on Unsplash

Math is beautiful, and the software we have at our fingertips represents an invaluable way to explore that beauty. Never before have we been able to visualize and experiment with mathematical objects in such an accessible way.

Today, I’d like to break down the steps required to plot a function of two variables using Python.

Along the way, we’ll learn about a few NumPy procedures that’ll have you manipulating matrices like a wizard. I’ve broken the process down into three conceptual steps that’ll also help refresh the underlying math.

Finally, I put everything together into a function you can use out of the box, so feel free to skip to that if it’s all you need. …


An intelligent framework for setting small goals

Image for post
Image for post
Photo by Frans Vledder on Unsplash

If you’re reading this, chances are that you already know what you need to do in order to improve your life.

You’re familiar with more than a couple life hacks at this point, and you know that building solid routines is important. You probably also know that it’s crucial to start small and not take on too much at once.

So why is it so difficult to just do the work? Why is it that we swing from not exercising at all to going to the gym 5 days a week, only to burnout and repeat the cycle? Notwithstanding all the advice on the internet telling us to start small, what makes the little voice in our heads saying “those rules don’t apply to you; you’re stronger than that” so compelling? …

About

Ray Heberer

Data Scientist at Proximate Research. Love thinking about AI, Python and R, Machine Learning, and Personal Growth.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store