Posts

Showing posts from 2018

Data analysis of developer job posts from Stack Overflow

Image
I did some data analysis on developer job posts from Stack Overflow with the aim of coming up with interesting insights about different aspects of job posts such as what are the most popular technologies, or the locations of job posts around the world as visualized on a map. After much data processing, I finally got the graphs I was looking for. I will show the most interesting graphs since there are too many to choose from and if you want to see the complete set of graphs, go to my personal website ; you will also find there explanations about how I managed to produce the data necessary to generate the graphs. The project is implemented in Python3 and makes use of a bunch of modules (e.g. BeautifulSoup4, matplotlib, plotly) for building the whole pipeline of generating the maps, graphs and reports. Contents 1. Sources of data and some initial stats 2. Bar chart: popularity of technologies based on number of occurrences in job posts 3. World map of job posts 4. Interactiv

Save HTML as a single file

I wanted to save some HTML pages as single files that could be read by Chrome and Firefox. I found out that you could do it in Chrome without any extension by enabling the experimental flag Save Page as MHTML . This experimental feature can be found by typing chrome://flags/#save-page-as-mhtml   in the search bar. However, Chrome warns you to be careful with playing with these options: ... you could lose browser data, or compromise your security or privacy. Once you enable this option, you must restart Chrome, and thereafter the files can be saved in the compressed format MHTML, " a single text file containing HTML and all sub-resources ". However, these MHTML files can be read by Chrome, but not by Firefox. Thus I searched for other tools that can generate HTML single files which can be read by both Chrome and Firefox, and these are the two extensions I tried that can accomplish this feat: SingleFile : Install for  Chrome  and Firefox Save Page WE : Install for  Ch

Python tips: reverse a dictionary

In Python, if you want to reverse a dictionary ( dict ), i.e. swap the keys and values, there are three methods that can help you achieve that goal depending on your needs: Method 1 : makes use of dict comprehension , and the dict must contain unique values Method 2 : makes use of dict.get() , and the dict doesn't contain unique values Method 3 : makes use of map(reversed, iter) , and you want the reverse dict to preserve the type and order of the original dict (if for example it is an OrderedDict ) I provided code implementations for each methods that work for Python 2.7.x and 3+, and the embedded code can be executed directly from the blog post through JDoodle , an online compiler and editor for many other languages (68 in total such as C/C++, Java, Go, SQL). I also compiled the average running times of the different methods in a table at the end ( Method 1 with dict comprehension is the fastest! Spoiler alert: Click to reveal winning method ) The code used

Modify DocumentRoot in Apache (Mac)

After you've installed Apache with Homebrew on macOS , you might want to change the DocumentRoot which points to the directory where your website resources reside (e.g. HTML, CSS, images). By default, DocumentRoot is located in /usr/local/var/www 1 . Here are the steps to follow to change DocumentRoot to map to a custom directory path: Place your website folder to the desired place. For example: /usr/local/var/www/my_website Open the file /usr/local/etc/httpd/httpd.conf with your favorite editor, search for DocumentRoot , and modify the following line by substituting your directory path within the double quotes: DocumentRoot "/your/directory/path" Also change the next line: <Directory "/your/directory/path"> Restart your Apache server (if you want Apache to restart at login): $ sudo brew services restart httpd or if you don't want Apache to run as a background service: $ sudo apachectl restart Test the changes to DocumentRo

Error: Keychain System cannot be found to store (Mac)

Image
After performing a Mac update, I found that my known Wi-Fi network was not joined automatically as I had specified in Network Preferences and this error message popped up: Keychain System cannot be found to store ... Also, this error message showed up while I did $ git push . My github login credentials stored within Keychain Access could not be accessed anymore. Solution I tried the first solution I stumbled upon which consisted in running First Aid on the logical drive Mac HD through Disk Utility ( Fig 1 ). And I restarted my computer (very important). Then my known network was now being joined automatically like before and the error message was not shown anymore! Fig 1. Select your Mac HD logical drive and run First Aid on it.

Heroku tutorials, definitions and useful commands

I followed the  PHP  and  Python  Heroku tutorials from their official website. While doing so, I saved some notes that might help you in installing the required software for using Heroku locally on your machine. These notes might assist you if you find yourself blocked while following one of the two tutorials because of an error message for example. Let's get started! NOTE:  the following notes make use of PHP 7.2.7, Python 3.6.4, Composer 1.6.5, and Heroku CLI 7.7.7 Contents 1. Notes for Tutorial 1: Getting Started on Heroku with PHP      1.1  Prerequisites      1.2 Install Composer (Linux/Unix/macOS)      1.3  Install PHP dependencies locally 2. Notes for Tutorial 2: Getting Started on Heroku with Python      2.1 Prerequisites      2.2  Install Python dependencies locally      2.3  Postgres issue 1: No manual entry for psql [macOS]      2.4  Postgres issue 2: could not connect to Postgres database 3. Important Heroku definitions 4. Heroku commands     4.1 Import

Install the MAMP (Mac, Apache, MySQL, PHP) stack

Image
There are two ways that you could install ( freely ) the whole MAMP stack: installing the MAMP (version 4.5) software bundle is the easiest way since it is a one-click-solution for setting up a personal webserver install each components (Apache, MySQL, and PHP) separately. This second method consists in installing Apache and PHP with Homebrew, and MySQL with the  package Installer . It is the hardest method but it enables you more control over the installation process, e.g you can install specific versions of each components, or install extra PHP extensions. NOTE:  MAMP 4.5, Apache 2.4.33, MySQL 8.0.11, PHP 7.2.7, and phpMyAdmin 4.8.2 are installed in the following tutorial Contents 1. Method 1: install the MAMP software bundle 2. Method 2: install each components (Apache, MySQL, and PHP) separately      2.1 Apache+PHP           2.1.1 Install Apache+PHP           2.1.2  Change Apache's default ports           2.1.3  Start the Apache server           2.1