Home > News > Wikipedia Down

Wikipedia Down

March 24, 2010 18:00 GMT: http://www.Wikipedia.org has been inaccessible for about 30 minutes now.

dig A http://www.wikipedia.org
http://www.wikipedia.org.      514     IN      CNAME   rr.wikimedia.org.

dig ANY http://www.wikimedia.org
http://www.wikimedia.org.      3600    IN      CNAME   rr.wikimedia.org.

dig ANY rr.wikimedia.org
there’s no entry for rr.wikimedia.org being published..

Also looks like ns2.wikimedia.org has fallen of the face of the earth and isn’t answering DNS queries.

ns2.wikimedia.org.      3600    IN      A       91.198.174.4

#nslookup http://www.wikimedia.org ns0.wikimedia.org

Server:         ns0.wikimedia.org
Address:        208.80.152.130#53

http://www.wikimedia.org       canonical name = rr.wikimedia.org.
rr.wikimedia.org        canonical name = rr.pmtpa.wikimedia.org.
Name:   rr.pmtpa.wikimedia.org
Address: 208.80.152.2

Update 18:30 GMT: In a tweet, Wikipedia confirmed and apologized for the site’s outage:

@Wikipedia; Sorry, everyone! English Wikipedia should be back in about an hour. Our tech team explains what happened: http://techblog.wikimedia.org/2010/03/global-outage-cooling-failure-and-dns/

Due to an overheating problem in our European data center many of our servers turned off to protect themselves. As this impacted all Wikipedia and other projects access from European users, we were forced to move all user traffic to our Florida cluster, for which we have a standard quick failover procedure in place, that changes our DNS entries.

However, shortly after we did this failover switch, it turned out that this failover mechanism was now broken, causing the DNS resolution of Wikimedia sites to stop working globally. This problem was quickly resolved, but unfortunately it may take up to an hour before access is restored for everyone, due to caching effects.

We apologize for the inconvenience this has caused.

Advertisements
Categories: News Tags: , ,
  1. March 24, 2010 at 6:22 pm

    Looks like not all DNS servers are affected. Google’s DNS server 8.8.8.8 returns info for rr.wikimedia.org

    # dig ANY rr.wikimedia.org @8.8.8.8

    rr.wikimedia.org. 578 IN CNAME rr.pmtpa.wikimedia.org.
    rr.pmtpa.wikimedia.org. 1390 IN A 208.80.152.2

  2. March 24, 2010 at 6:25 pm

    OpenDNS servers also return 208.80.152.2 for rr.wikimedia.org

    208.67.222.222
    208.67.220.220

  3. March 24, 2010 at 6:28 pm


    The TTL on the records from ns0.wikimedia.org:

    # dig ANY www,wikimedia.org @ns0.wikimedia.org

    www,wikimedia.org. 3600 IN CNAME rr.wikimedia.org.
    rr.wikimedia.org. 600 IN CNAME rr.pmtpa.wikimedia.org.
    rr.pmtpa.wikimedia.org. 3600 IN A 208.80.152.2

    So everything should be back to normal in 1 hours time.

    Note: www, should be www. above but wordpress likes to be smart and add http:// in front.

  4. March 24, 2010 at 6:33 pm

    So wikipedia.org is still there… the DNS is just a little screwed.

    # telnet 208.80.152.2 80
    Trying 208.80.152.2...
    Connected to 208.80.152.2.
    Escape character is '^]'.
    GET /wiki/Main_Page HTTP/1.0
    Host: en.wikipedia.org

    HTTP/1.0 200 OK
    Date: Wed, 24 Mar 2010 18:32:31 GMT
    Server: Apache
    Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
    Content-Language: en
    Vary: Accept-Encoding,Cookie
    Last-Modified: Wed, 24 Mar 2010 18:32:30 GMT
    Content-Length: 57295
    Content-Type: text/html; charset=UTF-8
    X-Cache: MISS from sq75.wikimedia.org
    X-Cache-Lookup: MISS from sq75.wikimedia.org:3128
    Age: 12
    X-Cache: HIT from sq37.wikimedia.org
    X-Cache-Lookup: HIT from sq37.wikimedia.org:80
    Connection: close

    Wikipedia, the free encyclopedia

  5. March 24, 2010 at 7:07 pm

    19:05 GMT – Wikipedia is back up. DNS has propagated. “All iz well” – to quote the 3 idiots

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: