Machine learning project review checklist

April 9, 2019 Matt Hall Environmental and Earth Sciences

Imagine being a manager or technical chief whose team has been working on a machine learning project. What questions should you be thinking about when your team tells you about their work?Here are some suggestions. Some of the questions are getting at reproducibility (for testing, archiving, or sharing the workflow), others at quality assurance. A few of the questions might depend on the particular task in hand, although I’ve tried to keep it pretty generic.There…

What makes a good benchmark dataset?

April 3, 2019 Matt Hall Environmental and Earth Sciences

An Ordance Survey benchmark. Last week I mentioned that we need more open benchmark datasets in geoscience. I think benchmarks are important for researchers to work on, as a teaching aid, and as a way for us to objectively measure how well we’re doing on a particular problem. How else can we know how we’re doing, or compare Company X’s claim with Company Y’s? What makes a good benchmark?I haven’t unearthed any guides from other…

Closing the gap: what next?

March 27, 2019 Matt Hall Environmental and Earth Sciences

I wrote recently about closing the gap between data science and the subsurface domain, naming some strategies that I think will speed up this process of digitalization.But even after the gap has closed in your organization, you’re really just getting started. It’s not enough to have contact between the two worlds, you need most of your actvity to be there. This means moving it from wherever it is now. This means time, and effort, and…

X lines of Python: Ternary diagrams

March 21, 2019 Matt Hall Environmental and Earth Sciences

Difficulty rating: beginer-friendly(I just realized that calling the more approachable tutorials ‘easy’ is perhaps not the most sympathetic way to put it. But I think this one is fairly approachable.)If you’re new to Python, plotting is a great way to get used to data structures, and even syntax, because you get immediate visual feedback. Plots are just fun.Data loadingThe first thing is to load the data, which is contained in a Google Sheets spreadsheet. If…

Closing the analytics–domain gap

March 15, 2019 Matt Hall Environmental and Earth Sciences

I recently figured out where Agile lives. Or at least where we strive to live. We live on the isthmus — the thin sliver of land — between the world of data science and the domain of the subsurface.We’re not alone. A growing number of others live there with us. There’s an encampment; I wrote about it earlier this week. Backman’s Island, one of my favourite kayaking destinations, is a passable metaphor for the relationship…

The digital subsurface water-cooler

March 12, 2019 Matt Hall Environmental and Earth Sciences

Back in August 2016 I told you about the Software Underground, an informal, grass-roots community of people who are into rocks and computers. At its heart is a public Slack group (Slack is a bit like Yammer or Skype but much more awesome). At the time, the Underground had 130 members. This morning, we hit ten times that number: there are now 1300 enthusiasts in the Underground!If you’re one of them, you already know that…

X lines of Python: Gridding map data

March 8, 2019 Matt Hall Environmental and Earth Sciences

Difficulty rating: moderate.Welcome to the latest in the X lines of Python series. You probably thought it had died, gawn to ‘eaven, was an x-series. Well, it’s back!Today we’re going to fit a regularly sampled surface — a grid — to an irregular set of points in (x, y) space. The points represent porosity, measured in volume percent.Here’s what we’re going to do; it all comes to only 9 lines of code!Load the data from…

The venue for TRANSFORM

February 27, 2019 Matt Hall Environmental and Earth Sciences

Last time I told you a bit about what to expect at the TRANSFORM unconference we’re hosting in May. But I haven’t really told you about the venue yet, and it’s one of the best bits. We’re hosting the event at the Château de Rosay, near Rouen in France. This is a large house in a small village. It is completely self-contained: we can sleep there, eat there, work there, relax there. There’s room for…

What’s happening at TRANSFORM?

February 15, 2019 Matt Hall Environmental and Earth Sciences

Last week, I laid out the case for naming and focusing on an open subsurface stack. To this end, we’re hosting TRANSFORM, an unconference, in May. At TRANSFORM, we’ll be mapping out the present state of things, imagining the future, and starting to build it together. You’re invited.This week, I want to tell you a bit more about what’s happening at the unconference.BYOS: Bring Your Own SessionWe’ll be using an unconference model. If you come…

The open subsurface stack

February 8, 2019 Matt Hall Environmental and Earth Sciences

Two observations:Agile has been writing about open source software for geology and geophysics for several years now (for example here in 2011 and here in 2016). Progress is slow. There are lots of useful tools, but lots of gaps too. Some new tools have appeared, others have died. Conclusion: a robust and trusted open stack is not going to magically appear. People — some of them representing large corporations — are talking more than ever…

Author: Matt Hall