How not to suck at GitHub

So you learned a bit of programming and you know how to use git. You are ready to materialize your thoughts through the mighty power of code, and make your art available to the whole world using the most popular open source social network on earth.

But you fear that you will make stupid mistakes and ruin your chance to become the next tech influencer. Rest assured, after reading this post you will know everything you need to know to make the most of Octocat’s home.

I will give you 10 tips to climb to the top 1% of GitHubbers faster than Pemba Dorje on oxygen!

1. Don’t look for alternatives

If you think of a novel utility, library or a plug-in, you need to start coding your own solution straight away.

You will only lose time looking for alternatives. You will find that prior art does not quite fit your requirements and, of course, you can not bend, even slightly, your problem definition to be solvable by existing solutions. Moreover, during this process, you may taint your original idea by looking at too many similar concepts, which will invariably worsen your design.

It would also be a huge waste of time trying to add a feature to an existing project. The legacy codebase of established projects is massive and messy. Just learning the relevant bits would take as much time as coding the whole solution from scratch anyway. And I’m not even talking about the near-infinite number of hours of work lost in feature requests which end up being rejected by the project owner after 3 months of silence.

2. Google and StackOverflow won’t help you

Don’t waste your precious time googling for a solution. Most of the answers on StackOverflow are garbage anyway, and the library you are working with (and which, of course, is the root cause of your current issue) is not that popular, hence chances of finding the exact solution to your problem online are minimal.

Instead, you better bother that project maintainer, whose lack of QA considerations is causing your troubles. The project maintainer has the mental map of his project engraved in his memory and therefore can figure out the solution to any problem near instantly. Plus, I suspect projet maintainers keep a secret list of known issues which they do not fix out of pure laziness and they actually wait until someone notices it to fix it.

3. Don’t try to fix it yourself

This tip is similar to number 2. If a project you are using doesn’t work, it’s probably the maintainer’s fault. Don’t even bother looking for an answer in the project wiki or investigating the bug yourself. Just go ahead and nag the project maintainer, he or she will probably be able to fix their buggy project just for you.
Don’t forget to check back each day and notify the maintainer if he forgot to respond to your last message.

4. Be as vague as possible

When you open an issue for a bug you’ve encountered (which should be about 2 minutes after you’ve noticed said bug if you follow the advice above), please do yourself a favor and describe your issue in as vague terms as possible. This way you avoid making a bad diagnosis and mislead the maintainer. Also, don’t post a snippet of your own code, system stats or installed libraries as it may leak sensitive information out to the public (remember hackers are constantly scraping GitHub for sensitive information!).

5. Don’t bother with the markup

You’ve heard about markdown but for your own sake, you couldn’t remember how to format code. Is it a <code> tag? Or maybe [code]? Or is it a back tick? Damn it!

Don’t worry, no one does. Just forget about formatting your issue altogether. Paste large blobs of code inline with your prose like you’re writing lyrics for the next trap banger. After all, if the maintainer is not happy with your format, he can copy and paste it to a text editor.
As a bonus, this way you don’t even need to indent your code properly!

6. Refrain from using reactions

It is outrageous that the GitHub team spent so much effort mimicking facebook with its stupid “reactions”. Ugh… Anyway, when you feel like you agree (or even disagree) with a comment, be respectful and post an additional comment to voice your opinion while adding absolutely no useful piece of information to the debate. And please, please, please, leave that stupid “thumbs up” button for social-network addicted kids.

7. DO NOT, ever, submit a pull request

Everyone knows project maintainers hate it when someone tries to touch their code. After all, why are they constantly finding excuses to reject pull requests?
Also, in order to preserve your ever so precious time, and avoid a painful public shaming, do not attempt to submit code to an open source project.

8. If you ever should submit code, don’t ask!

Let’s say there is this extremely useful feature you need, and the project seems a bit inactive lately. You really crave that feature and you can’t wait for Mr. slowpoke to code it himself. As per point 1, you do not have time to write the software from scratch yourself. You realize that it will be a substantial change which will require quite a bit of code, but nothing can discourage you. You need that feature.

Then make sure to secretly develop the feature without letting the maintainer know; otherwise, he might try to cut the grass under your feet. Don’t let anyone know you are working on this awesome addition until the very last moment when you submit your huge pull request as a single, solid, code change. Wow factor guaranteed!

9. Do not answer other people’s questions

As you contribute to repositories, you will notice other people try to drown your hard work under a deluge of irrelevant issues and comments. Don’t think helping them will make them go away; they will invariably come back with more.

Instead, keep a cool head and refrain from helping. If needed, “up” your own issue by asking for updates.

10. Use issues as a general purpose Q&A place

As we’ve covered in point number 2, you will not find answers on StackOverflow. What’s more, its elevator system tends to reorder comments in an annoying way which breaks chronological order.

Therefore, if you have anything to say, which is even remotely connected to a project, then go ahead and express yourself using that project’s issues. This will notify everyone who watches the repository and guarantee maximum exposure of your comment.


Hope you liked these tips. Please feel free to contribute your own in the comments!

How to manage and publish a multi-package TypeScript project

In the previous part, we’ve seen a few tricks which can help you split your TypeScript project into small packages. We were still missing a proper build system as well as a way to publish or deploy the packages. Which is what we will cover in this chapter.

To follow along

As in the previous part, I have created a simple repository which you can use to follow along this tutorial. This time however, you must first run a script to customize the project before you can start using the repository. The reason is that at the end of this tutorial, you will be able to publish your project to npm. In order to make this work for everyone, we have to rename the packages with a prefix which is unique to each reader of this tutorial. Don’t worry, it only takes a second:

  1. Download the archive for part 2, and unzip it somewhere.
  2. If you haven’t already, sign up to npm and log in using the command line. Or, if you don’t care about publishing to npm, go straight to 4.
  3. In the project, run ./customize.sh. If it worked, you can skip step 4.
  4. Otherwise, run the script like so: ./customize.sh username, where username is any username you picked. Beware that you will not be able to publish the packages.

Alternatively, if you do not wish to download the files, you can read the diff on github.

Build system

You may have noticed a new file called Makefile at the top of the project. Indeed, we use GNU Make to keep track of dependencies between packages and to define build tasks, and this file is where we tell Make what to do. If you do happen to know a better alternative than Make, please express yourself in the comments.

Make is installed by default on any decent operating system. If you are using windows, you can easily find instructions online to install it.

The top-level Makefile defines three things:

  • global tasks
  • dependencies
  • utility tasks

Global tasks

Global tasks are tasks which must run in all packages. They are defined in this line:

1
TASKS :=build clean test

For instance, the build task will build all packages at once. To run it, type:

1
make build

You will notice that this command also installs npm dependencies, and that it builds the packages in the correct order! Make did not magically figure this out, we instructed it to do so. Keep reading to find out how.

Dependencies

Dependency declaration is the easiest and most useful thing with Make. Simply write the path to the dependent, followed by a colon, followed by a space-separated list of dependencies.
For instance, the dependencies between our packages are defined like so:

1
2
3
packages/tstuto-api:
packages/tstuto-server: packages/tstuto-api
packages/tstuto-web-client: packages/tstuto-api

This is how Make knows in which order to build our packages. But how does it know to install npm dependencies first?

Open the Makefile in tstuto-web-client, which looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
BIN=public/dist/main.js
SRC =$(shell find src/ -type f -name '*.ts')
NODE_MODULES=node_modules/.makets
NPM_TASKS=test clean

.PHONY: build
build: $(BIN)

$(NODE_MODULES): package.json package-lock.json
npm install
touch $(NODE_MODULES)

$(BIN): $(NODE_MODULES) $(SRC)
npm run build

.PHONY: $(NPM_TASKS)
$(NPM_TASKS): $(NODE_MODULES)
npm run $@

The BIN variable contains the path to the output file of our package (in this case, it is the client-side javascript bundle).
We create a special file inside the node_modules directory, called .makets (which stands for make TimeStamp). This file records the last time that make ran npm install and helps it figure out if it should run the command again (that is, when either package.json or package-lock.json has changed).

Some tasks are defined in the script property of the package.json and do not need any additional logic in the Makefile. We define those in the NPM_TASKS variable which is used to run the npm script of the same name.

Utility tasks

Let’s go back to the top-level Makefile quickly to talk about utility tasks.

You might encounter repetitive tasks which are specific to one package and cannot be generalized to all packages. For instance, we want a task to start our development server. Such tasks can be defined individually in the top-level Makefile like so:

1
2
3
4
.PHONY: serve
serve:
$(MAKE) build
$(MAKE) -C packages/tstuto-server serve

Here we simply create a proxy-task which delegates to the makefile located in packages/tstuto-server.


That is about it for the build system. If you are familiar with Make, nothing should have surprised you in the above (except maybe the way we declare the dependencies for npm install). Otherwise, the syntax might look daunting at first, but you’ll quickly get used to it.

Without further ado, let’s see how we solve the last remaining problem: publishing.

Publishing/Deployment

The setup we have right now is great for development, but you may be wondering: “How do I deploy this to production?” or “How do I publish this as npm packages?”. Fear not, the answer lies right below!

You could of course clone your whole repository to your production environment and run make serve. That would work but it would also be quite unprofessional to proceed this way.

A more idiomatic way to proceed is to publish your packages to an artifact repository. Open source projects usually rely on npm’s public repository while proprietary software editors have their own private artifact servers. Luckily for us, this means that no matter what you are currently trying to do, whether it’s an open or closed source, whether it’s a library, a microservice or a CLI tool, the process to publish it is exactly the same!

Let’s recap what we want to do before digging into the details: We want to take all of our packages, give them appropriate version numbers and publish them using npm.
Oh, and one more detail: When they get published, our packages need to declare their dependencies to sibling packages in our project (because within the artifact repository, each package stands alone).

All the magic happens inside tools/publish.ts. This script does the following:

  • Determines the next version number
  • Changes all package.json files of all packages to set the correct version number
  • Adds the missing dependencies to each package.json
  • Performs a npm publish
  • Rolls back the changes made in the package.json files

I will not dive into the details of this script and I would not advise you to reuse it as-is for your own projects. Instead, you should try it out and understand how it works so you can apply this knowledge to your specific use-case.

We call this script from the Makefile. Run it with the following command:

1
make publish

If you are currently logged into npm, this will effectively publish the demo package to @username/tstuto-xxx.

You can now test your newly-deployed package by running:

1
npm install -g @username/tstuto-server

(Make sure to replace username with your actual npm username. You may need to run this command with sudo).

And then, start your server by running:

1
my-awesome-app

This should start the application.


Well done! If you made it so far, you now know the key elements to manage a multi-package TypeScript project, develop it and publish it (or ship it for production).

Bonus

Make can run multiple tasks in parallel in a way that is consistent with the dependencies declared in the makefile. For instance, to run up to 4 tasks in parallel, run make -j4 build. This can help you speed up builds when you have many packages and a flat dependency tree.

Tips and tricks to structure multi-package TypeScript projects

I’ve been experimenting with ways to structure a complex TypeScript project lately while working on Vaultage and I have finally found a solution that provides proper isolation and consistency across packages.

Java developers should be familiar with having dozens of sub-projects open simultaneously in their IDE, each with its own build target and dependencies.

In comparison, JavaScript projects tend to be a mess because of the lack of an idiomatic way to split code across packages. Tools like lerna help you create a proper JavaScript monorepo and rationalize the development of a complex project. However these tools pack a lot of features which take some time to properly master. The added complexity may result in misunderstanding and in the end the time gained by deploying the tool may be lost in obscure debugging sessions. Additionally, TypeScript is a different beast and requires more work to integrate properly.

In this tutorial, you will learn the basic principles behind a multi-package TypeScript project so you can apply them in your own work.

I created a minimalist project skeleton so you can follow along this tutorial. You can download it here.

The setup

Download and unpack the tutorial files into your workspace. You should end up with a project containing a packages sub-folder with three packages inside.
Each package is an independent unit of code, with its own build target and test suite:

  • The tstuto-server package contains the NodeJS server files
  • The tstuto-web-client package contains the web application which talks to the server
  • The tstuto-api package contains shared definitions which the client and server will use to communicate in a type-safe way.

As you may have guessed, our example application consists of a simple web server and a web application which talk over HTTP.

Go ahead and open the top-level folder in your favorite TypeScript IDE (it should be VSCode, if it’s not, then go ahead and download it now, I’ll wait…).

First, you’ll want to check that everything works as intended. Navigate to packages/tstuto-web-client and run:

1
2
npm install
npm run build

Then repeat this step for the tstuto-server package.

When you are done, you should have successfully built the demo application. Navigate to the tstuto-server package and run npm start to launch the server. Then, point your web browser at http://localhost:3000. You should see an ugly web page with a button.

Using a shared package

Take a look at the files packages/tstuto-server/src/controllers/MoodController.ts, and packages/tstuto-web-client/src/main-client.ts.

The client uses the axios library to fetch a mood from the server over HTTP. If you are a type safety freak, something should tickle your senses here: the communication is not safe. Indeed, look at the type returned by the axios call in main-client.ts: you’ll find that it is of the any type. You can use this object however you want and the TypeScript compiler will never complain, even though your code might crash at run-time!

An untyped response lets you type anything and provides no intellisense

Enter the tstuto-api package. Take a look at packages/tstuto-api/src/index.ts, we define a type and two factory functions there. Compile them by navigating to packages/tstuto-api and running npm install && npm run build.

Now, going back to MoodController.ts, replace the function by the following, which uses the factory methods instead of the inline object definitions:

1
2
3
4
5
6
7
8
9
10
11
12
13
import { happyMood, sadMood } from '../../../tstuto-api/src/index';
import * as express from 'express';

const HAPPY_THRESHOLD = 0.3;

export function MoodController(_req: express.Request, res: express.Response) {
const indicator = Math.random();
if (indicator > HAPPY_THRESHOLD) {
res.json(happyMood(indicator));
} else {
res.json(sadMood(indicator));
}
}

In main-client.ts, use the interface to type the object returned by axios:

1
2
3
4
5
6
import { IMoodAPIResponse } from '../../tstuto-api/src/index';

/* ... */

// line 23:
const mood = await axios.get<IMoodAPIResponse>('/api/mood');

That is way better, our API is now type-safe. The TypeScript compiler catches typos, and we have proper auto-completion and code refactoring!

A typed response provides completion and catches typos

However, written as is, our import statement actually instructs the TypeScript compiler to compile the target module along with ours. This has multiple nasty side effects and defeats the point of having separate modules altogether.

It would be much cleaner if we could just write:

1
import { IMoodAPIResponse } from 'tstuto-api';

We will see how we can achieve this in the next section.

Proper code sharing

So far, we’ve split our code into three modules and used import statements to borrow code from one module into another. However, we would like to isolate the modules further and import the built artifact rather than importing the raw source code. This means that we want the TypeScript compiler to use the type definitions emitted during compilation and we want node (or webpack for the client) to import the generated JavaScript file rather than the source TypeScript. This helps us avoid bugs spreading across modules and prevents careless developers from producing spaghetti code (to some extent…). It will also speed up your builds because tsc won’t have to compile the same source files over and over.

Go ahead and replace the two imports looking like import xxx from '../../api/src/index' in main-client.ts and MoodController.ts with just import xxx from 'api':

1
2
3
4
5
// main-client.ts:
import { IMoodAPIResponse } from 'tstuto-api';

// MoodController.ts:
import { happyMood, sadMood } from 'tstuto-api';

Don’t worry about the compiler error. All we need to do to make it disappear is instruct the TypeScript compiler to look for our custom packages in the packages folder. Fortunately, there is an option called baseUrl which allows us to do just that.

Edit tsconfig.json at the root of the project and uncomment the line "baseUrl": "packages". This way the TypeScript compiler also looks at the packages directory to resolve package names.

Note 1: Your packages may conflict with packages installed in node_modules; this is why we prefixed all our packages with tstuto-: to make sure that we don’t accidentally shadow an actual npm package.

Note 2: You may need to reload your editor after you changed tsconfig.json. In VSCode, open the command palette (CTRL+SHIFT+P) and chose “reload window”.

There is one more thing we need to do in order for this to work. TypeScript will not recognize your custom module unless you specified the type property in its package.json. Open packages/tstuto-api/package.json. You will see a line with the text “TODO”. Replace it with the following:

1
"types": "dist/index.d.ts"

You want to make extra sure that you got this setting right. If there is an error here, nobody will let you know, your imports might resolve to any and you won’t notice your mistake until it’s too late. The types property in package.json must point to the type definition of your entry point!

Now if you go back to packages/tstuto-server and run npm run build, you should get a successful build. However, if you try to start the server with npm run start, it will fail. Why? Although the TS compiler has figured out your project structure, Node.js is still oblivious to it: it doesn’t know where to find your custom modules at run time!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ npm start

> tstuto-server@0.0.0 start /workspace/ts-project-seed/packages/tstuto-server
> node bin/server.js

module.js:540
throw err;
^

Error: Cannot find module 'tstuto-api'
at Function.Module._resolveFilename (module.js:538:15)
at Function.Module._load (module.js:468:25)
at Module.require (module.js:587:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/workspace/ts-project-seed/packages/tstuto-server/dist/src/controllers/MoodController.js:3:20)
at Module._compile (module.js:643:30)
at Object.Module._extensions..js (module.js:654:10)
at Module.load (module.js:556:32)
at tryModuleLoad (module.js:499:12)
at Function.Module._load (module.js:491:3)

The next trick we will use is called NODE_PATH.

NODE_PATH is an environment variable node uses for pretty much the same purpose TypeScript uses “baseUrl”: it looks for additional node_modules inside the directory specified by NODE_PATH. The problem is that hacks based on environment variables tend to work poorly cross-platform. That’s why we will use cross-env, a nifty node module that lets you define environment variables in a portable way.

In packages/tstuto-server/package.json, replace the line "start": "node bin/server.js", with "start": "cross-env NODE_PATH=.. node bin/server.js",.

Now, when you run npm start, node will also look at the parent directory when resolving modules. This is how it will know that tstuto-api refers to your custom module in packages/tstuto-api.

Your application should now work properly!

Next steps

You have learned the fundamental tricks which will allow you to structure a multi-package typescript project. There are things left to fix though.

  1. It is tedious to go into each sub-project and manually run npm run build each time we change something. In addition, if we add more modules, manually tracking dependencies can quickly become a nightmare. That’s why we need a build and dependency tracking system to handle all of that for us.
  2. We would like to export our project, either to be distributed as an npm module or to be deployed somewhere. We can not ship our packages as separate npm packages just like that because they now depend on the directory structure of the repository.

See you in the next part of this tutorial, where we discuss those issues.


Appendix: How did you serve the client files again?

You may have noticed that our server also takes care to serve the client (as static files). While there are scenarios where you will want to ship the client separately, serving it from the API server is quite handy for development and suits a broad range of practical use-cases.

The trick fits into these three lines of code:

1
2
3
4
// Bind static content to server
const pathToWebUI = path.dirname(require.resolve('../../../tstuto-web-client'));
const staticDirToServer = path.join(pathToWebUI, 'public');
server.use(express.static(staticDirToServer));

We get the absolute path to the tstuto-web-client module and concatenate the public directory to it; we then instruct express to serve this folder as static content. Doing it this way allows us to keep the server and client completely separated and avoid any copy which would make our build system much more complex.

You should use the module name 'tstuto-web-client' instead of the relative path '../../../tstuto-web-client' now that you have learned the NODE_PATH trick. The reason the tutorial files ship with the relative path is to make it work as-is (even if you don’t set NODE_PATH), but this will break when you’ll try to deploy the app in the next part.


Thanks Ludovic for proofreading this tutorial.