For this answer, I will refer to querySelector and querySelectorAll as querySelector* and getElementById, getElementsByClassName, getElementsByTagName and getElementsByName are called getElement*.

Much of this information can be verified in the specification, and much of it was derived from various benchmarks I ran while writing. Specification: https://dom.spec.whatwg.org/

Main difference

1. querySelector* is more flexible because you can pass any CSS3 selector, not just a simple id, tag or class.
2. The performance of querySelector* varies with the size of the DOM it is called on. To be precise, querySelector* calls run in O(n) time, while getElement* calls run in O(1) time, where n is the total number of all child elements of the element or document on which the call is made.
3. The return types of these calls differ. querySelector and getElementById both return a single element. querySelectorAll and getElementsByName both return NodeList. getElementsByClassName and getElementsByTagName both return HTMLCollection. Both NodeList and HTMLCollection are called collections of elements.
4. Collections can return "live" or "static" collections respectively. This is not reflected in the actual type they return. The getElements* call returns a live collection, while querySelectorAll returns a static collection. From my understanding, live collections contain references to elements in the DOM, while static collections contain copies of elements. You can also check out @Jan Feldmann's comment below for a different perspective. I haven't found a good way to incorporate this into my answer, but it might be a more accurate understanding.

These concepts are summarized in the table below.

Function               | Live? | Type           | Time Complexity
querySelector          |       | Element        |  O(n)
querySelectorAll       |   N   | NodeList       |  O(n)
getElementById         |       | Element        |  O(1)
getElementsByClassName |   Y   | HTMLCollection |  O(1)
getElementsByTagName   |   Y   | HTMLCollection |  O(1)
getElementsByName      |   Y   | NodeList       |  O(1)

Details, Tips and Examples

• HTMLCollection is not as array-like as NodeList and does not support .forEach(). I found the spread operator useful to get around this problem:

  [...document.getElementsByClassName("someClass")].forEach()

• All these functions are accessible per element and globally document, except getElementById and getElementsByName, which are only available in document is implemented on.

• Chaining getElement* calls instead of querySelector* will improve performance, especially on very large DOMs. Usually faster even on small DOMs and/or very long chains. However, readability of querySelector* should be preferred over readability unless you know you need performance. querySelectorAll is usually harder to override because you have to select elements from a NodeList or HTMLCollection at each step. For example, the following code does not work :

  document.getElementsByClassName("someClass").getElementsByTagName("div")

  Because you can only use getElements* on a single element, not a collection, but if you only want one element then:

  document.querySelector("#someId .someClass div")

  can be written as:

  document.getElementById("someId").getElementsByClassName("someClass")[0].getElementsByTagName("div")[0]

  Note that [0] is used in each step of returning the collection to get the first element of the collection. The final result is only one element, just like using querySelector.

• Since all elements can be called using querySelector* and getElement*, it is possible to chain operations using both calls, which is useful when you want some performance improvements but can't avoid using getElement* calls that cannot be written in Very useful when querySelector.

• Although it is usually easy to tell whether a selector can be written using only getElement* calls, there is one situation where it may not be obvious:

  document.querySelectorAll(".class1.class2")

  can be rewritten as

  document.getElementsByClassName("class1 class2")

• Using getElement* on a static element obtained with querySelector* will cause the element to be dynamic relative to the static DOM subset copied by the querySelector, but static relative to the full document DOM... That's simple This is where the dynamic/static element interpretation starts to fall apart. You should try to avoid situations where you need to worry about this, but if it does exist, remember that querySelector* calls copy the elements they find before returning the reference, while getElement* calls get the reference directly without copying.

• querySelector* and getElementById traverse elements in a preorder, depth-first manner, called "tree order" in the specification. For the other getElement* calls, I can't tell from the spec if they are in the same tree order, but getElementsByClassName(".someClass")[0] may not result reliably in every browser. getElementById("#someId") should be reliable even if you have multiple copies of the same id on your page.

• I had to look into this issue when I was working on infinite scroll pages and I thought this might be a common situation where performance becomes an issue. We have onScroll event in our code which contains querySelectorAll call. Even if the calls are rate limited, the page will crash if you scroll far enough, at which point there will be too many calls iterating over too many elements for the browser to keep up. The size of the DOM is relevant in this use case, so in code running on an infinite scroll page, getElement* calls are preferred.

Syntax and browser support.

querySelector is more useful when you want to use more complex selectors.

For example, a list of all elements belonging to class foo: .foo li

: The characters have special meaning in selectors. You need to escape it. (Selector escape characters also have special meaning in JS strings, so you need to escape for it as well).

document.querySelector("#view\:_id1\:inputText1")