Our Blog

Photo by Björn Söderqvist, used under CC BY 2.0

Introduction

Whenever we like—or are forced!—to develop using a TDD (test-driven development) approach or a more traditional practice, unit testing should be always a part of our development process. And when it comes to .NET Core, no matter what framework we choose (xUnit, MSTest or other), we will probably need to use a mock library.

What is mocking? By definition, it’s making an imitation. Objects usually depend on other objects, which could rely on database connections, local files or external APIs. So when we need to test the behavior of a particular object, we will have to isolate its dependencies, replacing those objects with fake versions. And that’s where mocking—and the Moq library—comes into place: it’s a set of classes that allows us to easily create and handle those fake objects.

So, assuming we already have a .NET Core solution with a test project included, all we need to do to add Moq is install the package from the NuGet console in Visual Studio:

Install-Package Moq

Mocking data with async calls support

One of the first showstoppers I’ve encountered when trying to add unit tests to an existing project was to mock objects that contain asynchronous calls to the database: If we want to run offline (in-memory) tests against a set of code that performs an asynchronous query over a DbSet<T>, we’ll have to set up some helpers first.

So for example, let’s suppose we have a simple function called GetUserIDByEmail() that returns the ID of the user that matches an email passed by parameter, and that function uses an asyncronous query to the database to find that user:

• UserHandler.cs

public async Task<int?> GetUserIDByEmail(string Email)
{
    var User = await _DbContext.Users.Where(x => x.Email.Equals(Email)).FirstOrDefaultAsync();

    if (User != null)
        return User.ID;

    return null;
}

Where _DbContext is a reference to the interface IMockProjectDbContext, defined as:

• MockProjectDbContext...

Photo by Jon Flobrant on Unsplash

This post is a followup to my previous post, GraphQL — An Alternative to REST. Please check that out for an introduction to GraphQL and what makes it different from other API solutions. I’ve collected a list of some of the currently-maintained GraphQL libraries for a few different languages, along with some examples (most of which aren’t fully functional on their own, they’d need more configuration) so you can see what it might be like to use GraphQL in your project. I’ll be focusing on the ways each of these libraries implement GraphQL and what you’d need to do to start a project with each of them, so if you have questions about GraphQL itself, please check out my other blog post.

Apollo Server (JavaScript/TypeScript)

Apollo GraphQL has libraries for both a GraphQL server and client (which I’ll discuss later). Apollo Server can be used both as a standalone server as well as with libraries like Express. Apollo Server is the server library I have the most experience with—I wrote a server last year using Express and Apollo Server, along with a client that used Apollo Client. I’m a fan of the flexibiliy of Apollo, but it takes more work to set up than some of the alternatives.

Setting up Apollo Server as a standalone can be done fairly simply following the directions on their website. However, I’m going to go over the basics of integrating with Express. There are two main parts to writing a server with Apollo: your GraphQL schema and your resolvers. These stay more or less the same whether you’re using Apollo as a standalone server or combining it with Express. You do have to have a database set up separately; I’ll show examples with MongoDB, but you could easily swap it out with PostgreSQL or another database. I’ll show an example resolver along with its GraphQL schema for a blog post. The schema follows the GraphQL schema rules and might look like the following:

const typeDefs = [gql`
  type Post {
    id: String!
    body: String...

Photo by Sudhamshu Hebbar, used under CC BY 2.0

Earlier this year I wrote a post about my work with a machine-learning camera, the AWS DeepLens, which has onboard processing power to enable AI capabilities without sending data to the cloud. Neural networks are a type of ML model which achieves very impressive results on certain problems (including computer vision), so in this post I give a more thorough introduction to neural networks, and share some useful resources for those who want to dig deeper.

Neurons and Nodes

Neural networks are models inspired by the function of biological neural networks. They consist of nodes (arranged in layers), and the connections between those nodes. Each connection between two nodes enables one-way information transfer: a node either receives input from, or sends output to each node to which it is connected. Nodes typically have an “activation function”, parameterized by the node’s inputs, and its output is the result of this function.

As with the function of biological neural networks, the emergence of information processing from these mathematical operations is opaque. Nevertheless, complex artificial neural networks are capable of feats such as vision, language translation, and winning competitive games. As the technology improves, even more impressive tasks will become possible. As with organic brains, neural networks can achieve complex tasks only as a result of appropriate architecture, constraints, and training—for machine learning, humans must (for now) design it all.

Neural Network Architecture

Nodes are grouped in layers: the input layer, the output layer, and all the layers between them, known as hidden layers. Nodes can be networked in a variety of ways within and between layers, and sophisticated neural network models can include dozens of layers configured in various ways. These include layers which summarize, combine, eliminate, direct, or transform information. Each receives its input from the previous layer...

Image by Wikimedia Commons

In some sense, most machine learning projects look exactly the same. There are 4 stages to be concerned with no matter what the project is:

1. Sourcing the data
2. Transforming it
3. Building the model
4. Deploying it

It’s been said that #1 and #2 take most of ML engineers’ time. This is to emphasize how little time it sometimes feels the most fun part—#3—gets.

In the real world, though, #4 over time can take almost as much as the previous three.

Deployed models sometimes need to be rebuilt. They consume data that need to constantly go through points #1 and #2. It certainly isn’t always what’s shown in the classroom, where datasets perfectly fit in the memory and model training takes at most a couple hours on an old laptop.

Working with gigantic datasets isn’t the only problem. Data pipelines can take long hours to complete. What if some part of your infrastructure has an unexpected downtime? Do you just start it all over again from the very beginning?

Many solutions of course exist. With this article, I’d like to go over this problem space and present an approach that feels really nice and clean.

Project description

End Point Corporation was founded in 1995. That’s 24 years! About 9 years later, the oldest article on the company’s blog was published. Since that time, a staggering number of 1435 unique articles have been published. That’s a lot of words! This is something we can definitely use in a smart way.

For the purpose of having fun with building a production-grade data pipeline, let’s imagine the following project:

• A doc2vec model trained on the corpus of End Point’s blog articles
• Use of the paragraph vectors for each article to find the 10 other, most similar articles

I blogged about using the matrix factorization as a simple collaborative filtering style of the recommender system. We can think about today’s doc2vec-based model as an example of the content based filtering. The business value would be the potentially increased...

Photo by Kimberly Vardeman, used under CC BY 2.0

Ruby developers have access to a variety of linters and static program analysis tools that can greatly improve developer efficiency and code quality by catching syntax errors, detecting code smells, and making coding style suggestions based on popular style guides.

I have been a long-time advocate of configuring development environments with automatic code linting and will use this post to highlight some of the available tools for Ruby and methods for integrating them with popular code editors (Vim, Emacs, and Visual Studio Code).

Configuring your editor for automatic linting makes it much easier to identify and fix issues with your code at development time, and in-editor integration is very convenient for highlighting problems as you type (or save), making it easy to evaluate and improve the quality of your code as it is written.

Three popular linting plugins/extensions for Vim, Visual Studio Code, and Emacs are:

Asynchronous Lint Engine (ALE) Plugin for Vim

Provides asynchronous linting in Vim while you edit, and displays warnings/error messages in the editor. Supports the following tools for Ruby development and runs them automatically if they are found in your PATH:

• brakeman
• rails_best_practices
• reek
• rubocop
• ruby -wc (verbose syntax check)
• rufo
• solargraph
• standardrb

Ruby Extension for Visual Studio Code

Requires configuring the settings JSON file to enable each tool on an individual basis:

// Basic settings: turn linter(s) on
"ruby.lint": {
    "reek": true,
    "rubocop": true,
    "ruby": true, //Runs ruby -wc
    "fasterer": true,
    "debride": true,
    "ruby-lint": true
},

• debride
• fasterer
• reek
• rubocop
• ruby -wc (verbose syntax check)
• ruby-lint (unmaintained)

Flycheck Extension for Emacs

Detects and uses the following tools when editing Ruby code:

• reek
• rubocop
• ruby -wc (verbose syntax check)
• ruby-lint (unmaintained)

Linter Selection

I suggest starting small and only installing one or...

Photo by Jefferson Santos on Unsplash

This blog post is to alert the public, targeted individuals, and relevant authorities about a scam being perpetrated under the guise of a fictitious company called “EndPoint Petroleum Corporation”. The scammers have set up a website for this fake company, copying much of the content from Noble Energy. The scammers have adapted an old version of our logo for their nefarious purposes.

We will refrain from posting the URL for the scammers’ website since we don’t want to increase its SEO ranking, but it is shown in the image capture above. At first glance this website looks legitimate, but closer observation provides abundant evidence to the contrary. Rather than posting a list of its defects which would assist the scammers, I urge the interested reader to spend five minutes on the website.

An email forwarded to us from one of their targets provides an example of how the scammers are trying to use their website. They are targeting people outside the US with experience in the Oil and Gas Industry, telling them that they have been “shortlisted” for an Operations Manager position with the bogus company. The scammers then tell their targets to contact their “required travel agency” to make arrangements for a trip to the United States for an in-person interview.

We know that the scammers are using LinkedIn to contact their targets. They may also be contacting potential targets via other means as well. And it stands to reason that they may have posted their fake job offer on one or more job boards.

Trying to Take This Down

We have contacted and followed up with various authorities and organizations about this scam. We are going to continue our efforts in this regard. We welcome any assistance or advice that anyone can offer that would help with this. Please use our contact form to contact us.

Sample Scam Letters and Job Description

Below are several attachments that have been forwarded to us by one of their intended victims:

Photo by freestocks, used under CC0 1.0

I recently worked on getting a client’s Vue application converted to use Server Side Rendering (SSR). SSR works by generating the app’s initial HTML structure using a server-side process and then using client-side JavaScript to initialize or “hydrate” the application state on the client-side.

This worked out well in development and testing, however when it came time to roll things out to production, we ended up with a non-functioning application. While the server-rendered content was displaying inline, the client hydration piece was failing, and I was seeing browser console errors that I had never encountered before:

The error in Chrome:

chunk-vendors.342a7610.js:21 Uncaught DOMException: Failed to execute 'appendChild' on 'Node': This node type does not support this method.
...

The error in Safari:

[Error] HierarchyRequestError: The operation would yield an incorrect node tree.
    get (chunk-vendors.342a7610.js:27:30686)
    er (chunk-vendors.342a7610.js:27:30565)
    Le (chunk-vendors.342a7610.js:27:27806)
    xe (product-listing.0773c46e.js:1:18553)
[Error] TypeError: undefined is not an object (evaluating 't.$scopedSlots') — chunk-vendors.342a7610.js:27:27998
    on (chunk-vendors.342a7610.js:27:12025)
    rn (chunk-vendors.342a7610.js:27:11937)
    nn (chunk-vendors.342a7610.js:27:11584)
    (anonymous function) (chunk-vendors.342a7610.js:27:12759)
    fn (chunk-vendors.342a7610.js:27:12135)
    promiseReactionJob

Since the client hydration was failing, this resulted in a non-functioning front-end application which we had to quickly roll back. I’d scratched my head over this quite a bit, as this had been well-tested in development and staging.

After testing direct access to the origin server via /etc/hosts modification, we determined that the server itself was returning content that was working fine, so this pointed to something about the caching layer.

The errors being generated were similar to errors I’d...

Photo by Marco Verch, used under CC BY 2.0

Most Vue.js developers will be familiar with Vue.js devtools in the form of Chrome or Firefox browser extensions/addons. The Vue.js devtools integrate nicely into Chrome and Firefox’s native developer tools in the form of a new ‘Vue’ tab that provides a developer with the ability to browse their component hierarchy, investigate the state of their application’s Vuex store, and several other useful features.

I was a longtime Safari user who eventually became disappointed with its limited extension support (as compared to Chrome or Firefox-based browsers); there was once a Safari version of Vue.js devtools that required some manual installation, but development on that version ended back in early 2018.

An alternative for Safari users, or developers who want to debug Vue.js applications running on other clients such as mobile devices, is the standalone vue-remote-devtools app. vue-remote-devtools is an Electron app that runs in its own window and is loaded in a Vue.js application via a remote connection. I was curious to see how the developer experience of using vue-remote-devools compared to the Chrome browser extension that I was familiar with.

Installation of vue-remote-devtools was simple, with just a single yarn global add @vue/devtools command. Once installed, I was able to run the app with a vue-devtools command which opened a new window on my desktop:

Then it was a matter of updating my Vue.js application to connect to Vue.js remote devtools running locally on my laptop. I added the <script src="http://localhost:8098"></script> tag to my application’s index page as described in the vue-remote-devtools documentation. When I loaded my Vue.js application in Safari I was confronted with a new problem: a Content-Security-Policy error preventing the browser from connecting to local port 8098:

My Vue.js application is served from a Rails application via webpacker and already had a CSP in place, so it was relatively...