And now, I present my team 5 weeks into the season. Bolded players are players I either traded for or have picked up. Currently sitting at 0 wins and 4 losses
Mitchell - acquired as part of the Davis and Davis trade for Odell and Mitchell; jury is still out...
Odell - acquired as part of the Davis and Davis trade for Odell and Mitchell; will hold out on Odell; there's been a lot of interest + hoping the price goes up
Alex Collins - heard murmurs about Carson's neck injury and sent Collins to the top of my waiver adds list; I'm alright with the shift he put against the Rams
Jared Cook - Kittle back-up plan in 5, 4, 3, 2, 1...
Gainwell - picked up earlier this season; will see how things play out after numerous unsuccessful attempts to get Miles Sanders
Josh Reynolds - Aggie wide receivers need some love; Josh got a lot of targets with Julio and AJ Brown out but I don't think I'll hold on to him with Brown coming back
Mooney - missed out on Mooney in the draft but found him on the waivers earlier this season; I inexplicably dropped him then was able to pick him up again; could be worse...see Arob
Real Madrid win 1-0 after Mendy scores with a shot from outside the box late in the game. Atalanta never really threatened and Madrid playing again with a makeshift lineup. Madrid should have some key players back for the return leg.
]]>A tale of two halves. Elche really should've done better in the first half with the plethora of counterattacks that they had. FC Barcelona was forgiving in the first half but in the second half they were not.
1-1 draw.
]]>PSG earn a 4-1 victory!
]]>
Photo by Kaitlyn Baker on Unsplash
Documentation is essential to the adoption of open source projects as well as to the success of their communities. Season of Docs brings together technical writers and open source projects to foster collaboration and improve documentation in the open source space.
https://opensource.googleblog.com/2020/06/season-of-docs-now-accepting-technical.html
Bokeh
Improving the Documentation Experience for Bokeh Developers
Bokeh has done a tremendous job in documenting visualization use cases in the User Guide [1]. In the Reference [2], you can find all the API methods afforded by their models. The documentation has grown large and there is no easy way to find misspellings, repetition errors, or formatting issues in the text [3].
You can find dozens of code examples on how you might use Bokeh with your own data on GitHub[4]. You can find some of these examples inline in the documentation but not all of them are referenced[5]. Users may spend a considerable amount of time trying to figure out how a tool works without realizing there exists code they can reference. For example, you can use Themes to style a plot on Bokeh but these examples exist in the Reference when one would expect to find an example listed inline or referenced in the User Guide [6][7].
Lastly, a subset of the Bokeh documentation could benefit from the inclusion of metadata. Bokeh uses Sphinx to build documentation. Sphinx[8] is a tool that makes it easy to document software projects. This tool does not automatically include any structured data on the HTML pages it generates. Metadata in this case is metadata about the HTML pages. When searching for “Bokeh docs” on a search engine, the results users get back do not describe the content of the page. When sharing links to the Bokeh documentation on social media sites or forums, there is no way to preview the content on the page before clicking on links.
Vale Linter [9] is available as a GitHub Action [10]. It checks for spelling, repetition, and styling issues on every pull request. This Action can be added to the existing build process Bokeh uses for pull requests on GitHub. Automated checks would find existing errors in the documentation to fix. This technology would prevent future errors from creeping into the documentation. Vale Linter can also enforce a consistent writing style across all documentation. For example, suggesting the term "JavaScript" over "Javascript," preferring active voice over passive voice, etc.
Different parts of the documentation should link back and forth for a more complete discussion. Users interested in learning more about a topic should be able to navigate to the Reference from the User Guide. Users interested in seeing an example of an API method should also be able navigate to the User Guide from the Reference. All examples found in the GitHub repository should either be referenced or exist inline in the documentation.
Search engines extract and collate the metadata found on web pages to describe and classify them. Including metadata, such as descriptions, in the Bokeh documentation would give users more data when browsing search engine result pages. This metadata would also provide rich previews when sharing links to these pages. Some metadata would appear alongside these links, giving readers a preview of the content before clicking. Specifying HTML metadata, like a description, can be done by manually adding the the "meta" directive on some pages. Later, Sphinx extensions can be developed to automate adding relevant metadata throughout the entire documentation.
The goal of this tutorial is to build a search app using Django and Haystack You will learn how to use Django commands to initialize a database with emoji data. You will also learn how to add search to a Django project using Haystack.
Upon completion, you will have a built an app that allows you to search for over a thousand emojis. This app also gives you the ability to copy any emoji to your clipboard with one click.
Make sure you meet the following prerequisites before starting the tutorial steps:
Pipenv - version 2018.11.26 (run pipenv --version
to confirm)
Elasticsearch - version 5.5.3
This project depends on Pipenv. Pipenv allows you to download and install versions of packages in a virtual environment.
Another prerequisite is Elasticsearch. An Elasticsearch instance needs to run separate from the app.
The app depends on the following packages:
Django - to build the website
Haystack - to add search capability
Elasticsearch client - low-level client required by Haystack (different than the Elasticsearch instance)
Requests - to retrieve emoji data
Open up a terminal prompt and create a directory called emoji-in-the-haystack:
mkdir emoji-in-the-haystack
cd emoji-in-the-haystack
Install the packages:
pipenv install django==3.0.7
pipenv install git+https://github.com/django-haystack/django-haystack.git#egg=django-haystack
pipenv install elasticsearch==5.5.3
pipenv install requests==2.24.0
You’ll see a bunch of colorful output and a couple of 🐍 emojis. In this directory, you should now see the files Pipfile and Pipfile.lock.
You’re ready to create a Django project.
After installing the packages, the next step is to create a Django project.
Activate your virtual environment:
pipenv shell
You should now see your terminal prompt prefixed with
(emoji-in-the-haystack)
.
Create a Django project called emoji_haystack:
django-admin startproject emoji_haystack .
The directory should now look like this:
├── Pipfile
├── Pipfile.lock
├── manage.py
└── emoji_haystack
├── __init__.py
├── asgi.py
├── settings.py
├── urls.py
└── wsgi.py
Create a Django app called search:
python manage.py startapp search
The directory should now look like this:
├── Pipfile
├── Pipfile.lock
├── manage.py
├── emoji_haystack
│ ├── __init__.py
│ ├── asgi.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
└── search
├── __init__.py
├── admin.py
├── apps.py
├── migrations
│ └── __init__.py
├── models.py
├── tests.py
└── views.py
You need to enable the newly created app.
Update the INSTALLED_APPS
setting in settings.py:
33 34 35 36 37 38 39 40 41 42 |
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'search.apps.SearchConfig',
]
|
To test that everything is working, run the app:
python manage.py runserver
Navigate to http://127.0.0.1:8000/ and confirm that the app is working.
Note: You can run python manage.py migrate
to get rid of the Django
warnings when running the app.
The next step is to create a Django model class to represent the emoji data.
Update models.py:
1 2 3 4 5 6 7 8 9 10 |
from django.db import models
class Emoji(models.Model):
name = models.CharField(
max_length=50,
)
code = models.CharField(
max_length=50,
)
|
You need to store the name
for each emoji. For example, “grimacing
face” is the name given to 😬. You also need to store the code
for an
emoji. These code points are unique for every emoji. Django handles rendering
emojis in the browser using these codes.
After creating the model, run a migration to apply these changes to the database:
python manage.py makemigrations --name add_emoji_model search
python manage.py migrate
The next step is to create a new directory for the Django command. Django commands are special scripts registered in Django projects.
The command in this app retrieves emoji data and saves it to the database using
the Emoji
model class. This commands must live in the new directory.
Create the new directory:
cd search
mkdir management
cd management
mkdir commands
cd commands
Inside this commands directory, create the initemojidata command:
touch initemojidata.py
The directory should now look like this:
├── Pipfile
├── Pipfile.lock
├── db.sqlite3
├── emoji_haystack
│ ├── __init__.py
│ ├── asgi.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── manage.py
└── search
├── __init__.py
├── admin.py
├── apps.py
├── management
│ └── commands
│ └── initemojidata.py
├── migrations
│ ├── 0001_add_emoji_model.py
│ └── __init__.py
├── models.py
├── tests.py
└── views.py
Here is the code to retrieve and save emoji data:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
import json
import requests
from django.core.management.base import BaseCommand, CommandError
from search.models import Emoji
EMOJI_JSON_URL = 'https://raw.githubusercontent.com/iamcal/emoji-data/master/emoji.json'
class Command(BaseCommand):
help = 'Initialize database with emoji data'
def add_arguments(self, parser):
parser.add_argument(
'--dry-run',
action='store_true',
default=False)
def execute(self, *args, **options):
self.count = 0
try:
super().execute(*args, **options)
except KeyboardInterrupt:
self.stdout.write('')
self.stdout.write(self.style.SUCCESS(
'Emojis created: {}'.format(self.count)))
def handle(self, *args, **options):
self.dry_run = options['dry_run']
emojis = self.get_emojis()
for emoji in emojis:
if not emoji.get('name'):
continue
code = self.handle_code(emoji)
name = emoji['name'].lower()
self.stdout.write(
'{} - {}'.format(name, code))
if not self.dry_run:
emoji = Emoji(
name=name,
code=code)
emoji.save()
self.count += 1
def get_emojis(self):
response = requests.get(
url=EMOJI_JSON_URL)
emojis = json.loads(response.content)
return emojis
def handle_code(self, emoji):
"""
U+1F1EC, U+1F1FE - > 🇬🇾
"""
unified = emoji.get('non_qualified') or emoji.get('unified')
unified = unified.split('-')
codes = []
for code in unified:
_code = '&#x' + code
codes.append(_code)
return ''.join(codes)
|
The syntax for Django commands may take some time getting used to. Django
commands require a Command
class definition that subclasses
BaseCommand
. This class requires a handle()
method. Your logic
goes in here.
I use the execute()
method to
define some variables to count and output the number of items updated when a
command finishes running.
On line 35, the get_emojis()
method defined on the class gets called
using the self
property. The method makes a request to
the URL defined on line 9. This endpoint is a JSON file hosted
on GitHub.
It may not include the newest emojis but it’s the best option for this app. The Emojipedia API is no longer available for public use. Typically you need to handle errors when making API requests but it’s fine to leave out here.
The command retrieves the emoji data and begins to process each data item on
line 37. It ignores data items with no name
field. On line 41, the
command calls the handle_code()
. This method transforms the emoji
unicode data into a string that gets stored in the database. The transformation
of this unicode data makes it possible to render emojis in HTML. More on this
later.
You can run this command with an optional dry_run
argument. Providing
this argument means you can test your Django command logic without saving
anything to the database. If this argument is not passed in when running the
command, the command creates an Emoji object with name
and code
set and saves it to the database.
Django commands are ran from the root of the project.
Run the Django command (--dry-run
option):
python manage.py initemojidata --dry-run
Run the Django command (no regrets option):
python manage.py initemojidata
The emoji data is now stored in the database.
Haystack makes it easy to add custom search to Django apps. You write your search code once and can go back and forth between search backends as you please. You can choose to use different search backends like Elasticsearch, Solr, and others. This tutorial uses Elasticsearch.
Integrating Haystack consists of creating a search index model and updating a couple of Django settings.
The search index model corresponds to the database model defined earlier. Haystack requires this file to know what data to place in the search index.
Inside the search app directory, create a search_indexes.py file:
cd search
touch search_indexes.py
Here’s what the code for that looks like:
1 2 3 4 5 6 7 8 9 10 11 |
import datetime
from haystack import indexes
from search.models import Emoji
class EmojiIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
def get_model(self):
return Emoji
|
When you make search a query, Haystack searches the text
field. This
field corresponds to the name
field defined in the Emoji
model.
Next, include the urls provided by Haystack in urls.py. Django implicitly calls a custom Haystack view that handles search requests and returning responses. This response uses an HTML template that you need to create and configure. More on this later.
16 17 18 19 20 21 22 |
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path('admin/', admin.site.urls),
path('search/', include('haystack.urls')),
]
|
You need to enable the Haystack app.
Update the INSTALLED_APPS
setting in settings.py:
33 34 35 36 37 38 39 40 41 42 43 44 |
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'search.apps.SearchConfig',
'haystack',
]
|
Add a connection to Elasticsearch in settings.py:
127 128 129 130 131 132 133 134 135 136 |
# Haystack configuration
# https://haystacksearch.org
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch5_backend.Elasticsearch5SearchEngine',
'URL': 'http://127.0.0.1:9200/',
'INDEX_NAME': 'haystack',
},
}
|
The following steps are cumbersome but they are essential in getting Haystack to work.
In settings.py, update the TEMPLATES
setting:
58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR, 'templates')],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
|
From the root of the project, create a templates directory:
mkdir templates
cd templates
Creating a single project-level templates directory is a recognized Django pattern.
In the templates directory, create a search directory and a file called search.html:
mkdir search
cd search
touch search.html
In the search directory, create an indexes directory:
mkdir indexes
cd indexes
In the indexes directory, create a search directory and a file called emoji_text.txt:
mkdir search
cd search
touch emoji_text.txt
Here’s what emoji_text.txt should look like:
{{ object.name }}
Haystack uses this data template to build the document used by the search engine.
The final directory structure should look like this:
├── Pipfile
├── Pipfile.lock
├── db.sqlite3
├── emoji_haystack
│ ├── __init__.py
│ ├── asgi.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── manage.py
├── search
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── management
│ │ └── commands
│ │ └── initemojidata.py
│ ├── migrations
│ │ ├── 0001_add_emoji_model.py
│ │ └── __init__.py
│ ├── models.py
│ ├── search_indexes.py
│ ├── tests.py
│ └── views.py
└── templates
└── search
├── indexes
│ └── search
│ └── emoji_text.txt
└── search.html
Now it’s time to update search.html. This template contains a text field to type in a search query, a button that fires a search request and some template variables. Use the template example found here.
Note: Remove {% extends 'base.html' %}
at the top of the file.
The main differences in the template for this tutorial are the following two lines:
18 19 20 21 |
{% for result in page.object_list %}
<p>{{ result.object.code|safe }}</p>
<p>{{ result.object.name }}</p>
{% empty %}
|
object_list
is a list of search results. For each search result, display
the emoji and its name. result.object
provides direct access to the
Emoji
model and its database fields.
Displaying the emoji requires using the safe
Django filter. It does not
require further HTML escaping.
Navigate to the location of your Elasticsearch installation and start an instance. For example, say you downloaded Elasticsearch in your Downloads folder:
cd Downloads
cd elasticsearch-5.5.3
cd bin
elasticsearch
Haystack ships with a set of Django commands that handle indexing the emoji data stored in the database. This tutorial uses the rebuild_index command. This command rebuilds the search index by first clearing it and then updating it. Have a look at the source code for more info.
From the root of the project, run the command:
python manage.py rebuild_index
Run the app:
python manage.py runserver
Navigate to http://127.0.0.1:8000/search and confirm that the app is working.
If you query for “cat,” you get back a list of results. If you query for “flag,” you get back results for flag emojis.
If you scroll to the bottom, you’ll see a Previous button and Next button. Haystack returns at most 20 results per page. This out of the box feature is awesome. The layout needs a little bit of work though.
You can use Bootstrap to clean up the design. Another feature is to copy an emoji to your clipboard by clicking on it - clipboard.js can help here.
Load Bootstrap and clipboard.js from CDN in search.html:
1 2 3 4 5 6 7 8 9 |
<script src="https://cdn.jsdelivr.net/npm/clipboard@2/dist/clipboard.min.js"></script>
<!-- Bootstrap CSS -->
<link rel="stylesheet"
href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css"
integrity="sha384-MCw98/SFnGE8fJT3GXwEOngsV7Zt27NXFoaoApmYm81iuXoPkFOJwJ8ERdknLPMO"
crossorigin="anonymous">
{% block content %}
|
A couple of Bootstrap <div>
elements and some styling updates go
a long way in improving the look of the app.
Including the data-clipboard-text
attribute on the emoji button lets
you copy emojis to your clipboard:
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
{% if query %}
<h3>Results</h3>
<div class="container">
<div class="row">
{% for result in page.object_list %}
<div class="col-sm">
<button type="button" class="btn" data-clipboard-text="{{ result.object.code|safe }}" style="font-size:90px;">{{ result.object.code|safe }}</button>
<p style="text-align: center">{{ result.object.name }}</p>
</div>
{% empty %}
<p>No results found.</p>
{% endfor %}
</div>
</div>
|
The last thing to do is to initialize clipboard.js in search.html:
50 51 52 53 54 55 56 57 58 59 60 61 62 |
{% endblock %}
<script>
var clipboard = new ClipboardJS('.btn');
clipboard.on('success', function(e) {
console.log(e);
});
clipboard.on('error', function(e) {
console.log(e);
});
</script>
|
Run the app with these new changes:
python manage.py runserver
Navigate to http://127.0.0.1:8000/search and confirm the changes. This looks much better. The emojis are more prominent and the click-to-copy feature is the 🍒 on top.
Rejoice and show your friends how to find the emoji in the haystack. If you’re up for the challenge, see if you can make the following app improvements:
Load a subset of emojis on the homepage before a user searches
Add a navigation bar to filter by emoji category
Support for newer emojis
Webhooks - less painful than playing hooky by skipping work.
Image source: Encyclopedia SpongeBobia
key
is set to the secret token encoded in bytesmsg
is set to the request body encoded in bytesdigestmod
is set to the SHA-1 hashing algorithm==
operator when comparing values in Python. Do not do this here! Heed the following warning found in the Python documentation:Warning: When comparing the output ofdigest()
to an externally-supplied digest during a verification routine, it is recommended to use thecompare_digest()
function instead of the==
operator to reduce the vulnerability to timing attacks.
I've been a Posthaven user for less than a week. Here's what I've gathered about the platform:
Larry David knows all about phone etiquette.
Image source: NBC News
I recently worked on improving some phone number validation logic at Winnie. We validate a batch of phone numbers and send them off to a third-party service. Some of the numbers we were sending were deemed invalid by the service. This was preventing us from automating some data updates we wanted to run daily. How hard could validating digits be?
Some validation boxes we already checked off included:
An edge case we were not considering were 800 numbers! A code change went out to ignore these type of phone numbers. The next day we were able to send a new batch of phone numbers to the third-party with no issues. Problem solved? Not quite.
We were still sending them invalid phone numbers. Perfectly-looking phone numbers were being deemed invalid by them. For example, 234-911-5678 is an invalid phone number. How? There are no non-digit characters and it looks like a valid phone number!
It turns out there is something called the North American Numbering Plan. Under the modern plan, a U.S. phone number must adhere to the NPA-NXX-xxxx format. Each component in this format must follow certain rules. The valid range for the first digit in the NPA component is 2-9. The valid range for each digit in the xxxx component is 0-9. 123-234-5678 is invalid because the first digit is a 1. In the example above, 234-911-5678 was invalid because it violated the following rule: the second and third digit in the NXX component can't both be 1.
I was determined to avoid translating these rules to brittle Python code. I knew there had to be a solution we could leverage instead of reinventing the wheel.
What does a software engineer do when they're stuck? Turn to Google. Here are some search queries I tried:
def is_valid(self):
"""
checks whether the number supplied is actually valid
"""
return phonenumbers.is_valid_number(self)
is_valid_number
to get the method definition but got nothing. I realized that phonenumbers was an external package that the project relied on. I immediately Googled the package, skimmed the README and tested it with our invalid phone numbers. It worked! I was confident that this package was enough for our needs and soon it found a home in our requirements.txt file.
Would I have saved 5 minutes by skipping straight to the README instead of browsing the source code? Sure. But being able to read code, especially code that you didn't write, is a useful skill. Plus, GitHub has made it even easier to navigate code on their platform. Can you tell I'm a fan of GitHub?
Googling is a skill. Reading source code is a skill. Reading documentation is a skill. Combine these skills with communicating effectively and what do you get? Probably something better than a 10x engineer.