Still seeing ongoing intermittent Cloudflare issues where things seem to work for 5-10 minutes then go down for 5-10 minutes. This is a global outage affecting many (if not all) websites that use it.
E-Hentai
E-Hentai is an image-hosting and file-sharing website focused on hentai (Japanese cartoon pornography). The site hosts user-generated image galleries primarily of pornographic content originating or derived from anime, manga, and video games, such as fanart, scanlations of manga and dōjinshi, and cosplay photographs. Its sister site Exhentai (colloquially known as Sad Panda) was spun off from E-Hentai in 2010 to host artwork that depicts material that is illegal in certain jurisdictions, such as lolicon, shotacon, and bestiality. More on Wikipedia
-
Tenboro
@ehentai
-
Tenboro
@ehentai
We are seeing intermittent issues with Cloudflare resulting in a Cloudflare error screen. No issues on our end, and I'm sure they are working on it.
-
Tenboro
@ehentai
The routing outage appears to be resolved, let us know on the forums if you are still having problems.
-
Tenboro
@ehentai
There is a partial routing outage affecting one of our locations, which means some services like HV and the forums are currently unavailable.
-
Tenboro
@ehentai
We are seeing a lot of spam from old accounts reusing passwords that have been leaked elsewhere, so unless you want your account to be terminated, make sure to use a strong and unique password. You won't get the account reinstated if this happens to you, which it will eventually.
-
Tenboro
@ehentai
One of the four main frontends experienced some sort of issue causing it to enter a hung state, which would have caused various issues. It has now been recovered.
-
Tenboro
@ehentai
The downed servers have all been recovered and linked back up, and everything appears to be operating normally. Let us know if you encounter any issues.
-
Tenboro
@ehentai
There was a power failure at one of our locations, which is causing some issues. Some services will be unavailable until all servers have been restored.
-
Tenboro
@ehentai
New banners posted. Thanks to arashileung, Look A Moth, The Magi, Honeycat, sparroff, Noni, Necromusume, 多路卡, 绳系胖次, mathl33t, Ovan Kura Majutsu, Acetominaphene, awesomeman01 and Boboosh.
-
Tenboro
@ehentai
The downtime-incurring part of the maintenance has been completed, all services should be back online. Let me know if on the forum if something is broken.
-
Tenboro
@ehentai
Unfortunately we will need to do a full restart cycle on the database servers to fix this, so the various parts of the site will soon be brought offline or put in read-only mode. There are quite a lot of database servers so maintenance will probably take at least an hour or two.
-
Tenboro
@ehentai
The wiki will be down for a bit for some maintenance. Also, due to an internal database problem we may have to restart some or all the site's databases some time later today, which would require some downtime/read-only.
-
Tenboro
@ehentai
Banners updated. Thanks to Noni, The Magi, Look A Moth, Ovan Kura Majutsu, Acetominaphene, mathl33t, 绳系胖次, Boboosh, arashileung, Necromusume, 多路卡 and Honeycat.
-
Tenboro
@ehentai
News banners updated. Thanks to Noni, Acetominaphene, Honeycat, arashileung, Look A Moth, The Magi, 绳系胖次, Necromusume, Ovan Kura Majutsu and 多路卡.
-
Tenboro
@ehentai
Banners updated. Thanks to Honeycat, Noni, ezequiell, Ovan Kura Majutsu, Look A Moth, Acetominaphene, The Magi, Boboosh, 绳系胖次, Necromusume, mathl33t, arashileung and 多路卡.
-
Tenboro
@ehentai
New banners posted. Thanks to sparroff, Acetominaphene, 绳系胖次, 多路卡, Honeycat, Look A Moth, mathl33t, Necromusume, Ovan Kura Majutsu, arashileung, Giodora Sue, Noni and The Magi.
-
Tenboro
@ehentai
New banners up. Thanks to Ovan Kura Majutsu, Noni, The Magi, SomeFuckingFaggot, mathl33t, Honeycat, dongmian, Necromusume, uyrth02 and Look A Moth.
-
Tenboro
@ehentai
The networking issues seem to have been resolved.
-
Tenboro
@ehentai
We're experiencing some sort of network issue partially affecting two of the site's frontends. Traffic has been rerouted for now, currently investigating.
-
Tenboro
@ehentai
Banners updated, thanks to Look A Moth, Noni, dongmian, 樱樱樱, Ovan Kura Majutsu, uyrth02, 多路卡, Honeycat, Necromusume, mathl33t and Sadao Maou. Oh, and happy new year.
-
Tenboro
@ehentai
Banners updated. Thanks to Look A Moth, dongmian, Noni, Ovan Kura Majutsu, mathl33t, Sadao Maou, 多路卡, uyrth02, canburendun and Necromusume.
-
Tenboro
@ehentai
We'll complete the deployment of the new search engine in a little bit. You may see some brief glitching as it's been deployed, but no downtime is expected, unless something explodes.
-
Tenboro
@ehentai
We will soon start a staggered deployment of a new version of the site, featuring a brand new search engine. No downtime is expected, but since navigation has changed you can expect to be punted back to the first page of a search result if you are in the middle of browsing one.
-
Tenboro
@ehentai
News banners updated. Thanks to Pretty anon, registered member, Look A Moth, Necromusume, Noni, Ovan Kura Majutsu, mathl33t, 多路卡, Sadao Maou and uyrth02.
-
Tenboro
@ehentai
All the servers have been recovered, and performance should be back to normal.
-
Tenboro
@ehentai
All public services should now be restored, through I still need to recover a bunch of databases to restore full redundancy/performance.
-
Tenboro
@ehentai
There was a power transient at our main datacenter that caused several important servers to go down with no warning. Working on recovery, but this will take some time.
-
Tenboro
@ehentai
Some sort of failure caused a server used by among other things HV and the forums to crash and reboot, currently looking into it.
-
Tenboro
@ehentai
Banners updated. Thanks to sparroff, registered member, Noni, Look A Moth, mathl33t, Necromusume, Ovan Kura Majutsu, Pretty anon, uyrth02, amaimono, Sadao Maou and 多路卡.
-
Tenboro
@ehentai
The PSU on the failed frontend has been replaced and recovery has been completed. It should now be fully back online.
-
Tenboro
@ehentai
One of the site's frontends has encountered some sort of critical power-related hardware failure and has been taken offline until this is resolved. Some archives will be unavailable until it is back online, but everything else should operate just fine off the remaining ones.
-
Tenboro
@ehentai
News banners updated. Thanks to amaimono, Look A Moth, Pretty anon, mathl33t, Ovan Kura Majutsu, WilyMason, Sadao Maou, Noni, 多路卡, registered member, Necromusume and uyrth02.
-
Tenboro
@ehentai
News banners updated. Thanks to Look A Moth, Sadao Maou, Noni, Ovan Kura Majutsu, amaimono, mathl33t, Pretty anon, Mags_, registered member, 多路卡 and Necromusume.
-
Tenboro
@ehentai
Wiki maintenance was completed.
-
Tenboro
@ehentai
The wiki and HV alt will be down for about half an hour for some updates.
-
Tenboro
@ehentai
News banners updated. Thanks to Noni, Look A Moth, 多路卡, sparroff, Pretty anon, mathl33t, oneer, Sadao Maou, registered member, amaimono, Necromusume, Ovan Kura Majutsu and Shizuka Natsuki.
-
Tenboro
@ehentai
Traffic has been stable for the last 30 minutes or so, so the problem seems to have been resolved.
-
Tenboro
@ehentai
Cloudflare has confirmed system-wide outages, awaiting resolution. www.cloudflarestatus.com
-
Tenboro
@ehentai
Investigating an issue causing 503s on parts of the site. Looks like a Cloudflare problem, but not entirely sure at this point.
-
Tenboro
@ehentai
News banners updated. Thanks to Look A Moth, Pretty anon, 多路卡, Noni, sparroff, Ovan Kura Majutsu, Necromusume, amaimono, Sadao Maou and oneer.
-
Tenboro
@ehentai
News banners updated. Thanks to Srac, Pretty anon, mathl33t, Ovan Kura Majutsu, Look A Moth, Necromusume, 多路卡, Sadao Maou, Noni and wenx595.
-
Tenboro
@ehentai
There seems to be some significant disruptions at a core IX/carrier which may be affecting various parts of the site depending on your location.
-
Tenboro
@ehentai
You may be seeing some disruptions caused by upstream routing issues. Host has been alerted.
-
Tenboro
@ehentai
News banners updated. Thanks to Noni, 多路卡, Sadao Maou, Ovan Kura Majutsu, Look A Moth, Pretty anon, Adhinferno Bloodmoon and Necromusume.
-
Tenboro
@ehentai
The frontend maintenance has been completed.
-
Tenboro
@ehentai
The frontends will be undergoing some maintenance over the next few days. This should not cause any downtime, except that archives assigned to a particular frontend won't be available while it's being worked on.
-
Tenboro
@ehentai
The wiki/etc maintenance was completed.
-
Tenboro
@ehentai
The wiki and HV alt will be down for 30-60 minutes for some host maintenance.
-
Tenboro
@ehentai
The forum maintenance has been completed.
-
Tenboro
@ehentai
The forums will be unavailable for up to an hour or so for some maintenance. Account logins will also be unavailable during this.
-
Tenboro
@ehentai
We are seeing some significant routing issues at a core location, which is going to be causing issues for large parts of the site until it is resolved.
-
Tenboro
@ehentai
New banners up. Thanks to Look A Moth, Noni, 多路卡, Adhinferno Bloodmoon, Pretty anon, Ovan Kura Majutsu and WheldrakeX.
-
Tenboro
@ehentai
The frontend has been repaired and is fully back online. The issue was faulty optics in the switch port or cable causing the 10gbit uplink to degrade to 100mbit.
-
Tenboro
@ehentai
Due to a network issue on one of the frontends, you may see some slowness and some archives being temporarily unavailable.
-
Tenboro
@ehentai
Wiki maintenance was completed.
-
Tenboro
@ehentai
The wiki will be offline for half an hour or so for some updates.
-
Tenboro
@ehentai
The nodes on the first shard have all been swapped. Let me know on the forums if you notice any weirdness.
-
Tenboro
@ehentai
We'll be swapping out the cluster nodes on the first of three web shards over the next couple of hours. This is not expected to cause any disruption, but as it is a major version update of everything, some glitching may occur.
-
Tenboro
@ehentai
Banners updated. Thanks to Pretty anon, Look A Moth, Frostbite, Noni, Ovan Kura Majutsu, mathl33t and Adhinferno Bloodmoon
-
Tenboro
@ehentai
New banners are up; thanks to Look A Moth, Frostbite, Pretty anon, Frozendat, Noni, SIRIUSs, Spectre and mathl33t. Oh, and happy new year.
-
Tenboro
@ehentai
Since a lot of people are asking through a lot of different channels; no, H@H does not include Log4j and is not affected by the recent vulnerability.
-
Tenboro
@ehentai
News banners updated. Thanks to Pretty anon, Noni, Frozendat, Look A Moth, Frostbite, Celibate_Wanker, mathl33t and SIRIUSs.
-
Tenboro
@ehentai
New banners were posted. Thanks to Frozendat, Pretty anon, Look A Moth, Spectre, Noni, Frostbite and Sadao Maou.
-
Tenboro
@ehentai
After a beta test that lasted for about a month, new uploaders are now officially "live" and will handle all new gallery uploads. Old unpublished galleries will be migrated over soon, but are still managed on the old uploaders for now.
-
Tenboro
@ehentai
Banners updated. Thanks to Frozendat, Pretty anon, Look A Moth, NBRa, Noni, Frostbite, lttcyber and Sadao Maou.
-
Tenboro
@ehentai
News banners updated. Thanks to NBRa, Frostbite, Paarfi, Frozendat, Pretty anon, Noni, Look A Moth and Try_.
-
Tenboro
@ehentai
News banners updated. Thanks to NBRa, Look A Moth, Frostbite, Try_, Noni, Paarfi, Pretty anon, Frozendat and Ekusas00.
-
Tenboro
@ehentai
The wiki will be unavailable for ten minutes or so for some updates.
-
Tenboro
@ehentai
News banners updated. Thanks to NBRa, Look A Moth, Noni, Frozendat, Try_, WheldrakeX, Pretty anon, Frostbite and Paarfi.
-
Tenboro
@ehentai
News banners updated. Thanks to Noeleo, Noni, Look A Moth, Frostbite, Spectre, WheldrakeX and Pretty anon.
-
Tenboro
@ehentai
News banners updated. Thanks to Frostbite, WheldrakeX, Look A Moth, Pretty anon, Noeleo, Noni, sdpfsj and Grandmasters.
-
Tenboro
@ehentai
News banners updated. Thanks to Pretty anon, Srac, Frostbite, Noni, Look A Moth, sdpfsj, Paarfi and Noeleo.
-
Tenboro
@ehentai
The forum was briefly unavailable due to a routing issue at the host, which has been resolved.
-
Tenboro
@ehentai
News banners updated. Thanks to nasu, Noni, Frostbite, sdpfsj, Noeleo, Pretty anon, Look A Moth, Someguywith and Paarfi.
-
Tenboro
@ehentai
We'll be taking one of the core servers down to replace some faulty RAM. You may see some occasional errors as things are swapped over to the other cores.
-
Tenboro
@ehentai
The switch was completed and everything appears to be functioning. Let me know if you encounter any new issues.
-
Tenboro
@ehentai
The forum and account logins will be briefly unavailable while we switch it to a different server.
-
Tenboro
@ehentai
The switch has been replaced, and all frontend servers should now be back online.
-
Tenboro
@ehentai
Looks like there was a switch failure in the rack these particular servers are located in. Still awaiting final resolution from the host.
-
Tenboro
@ehentai
Traffic has been rerouted to the working frontends. Some archives will not be available, and you may see slower than normal loads until it's back to normal.
-
Tenboro
@ehentai
There is a network outage at our host affecting some of our frontends, which is causing various issues.
-
Tenboro
@ehentai
News banners have been updated. Thanks to Frostbite, Pretty anon, Noeleo, Spectre, sdpfsj, Someguywith and Look A Moth.
-
Tenboro
@ehentai
We'll be switching over the "repo" services which handle torrent and bounty creation functionality to new servers shortly. You may briefly get some missing torrent errors and similar until everything has been switched over.
-
Tenboro
@ehentai
The maintenance has been completed, and everything should be back up. Please report any issues on the forums.
-
Tenboro
@ehentai
The maintenance will start shortly.
-
Tenboro
@ehentai
We are planning some database maintenance tomorrow, January 20th, starting at around 08:00 UTC. Galleries should remain available in a basic read-only mode, but various functionality will be unavailable or degraded during the maintenance.
-
Tenboro
@ehentai
All the affected secondary services should now be available again.
-
Tenboro
@ehentai
Some secondary services like the wiki and HV alt will be unavailable for a little while due to some host maintenance. Expected downtime is around 30 minutes.
-
Tenboro
@ehentai
Happy New Year, and happy new news banners. Thanks to Pretty anon, xxxhentaii, Noni, WheldrakeX, sdpfsj, Noeleo, Look A Moth and Frostbite. Kotoyoro~
-
Tenboro
@ehentai
News banners updated. Thanks to Look A Moth, Honeycat, Pretty anon, Noeleo, sdpfsj, Spectre and DJNoni.
-
Tenboro
@ehentai
The wiki has been migrated and should be back online. Note that it could take a short time for DNS records to update. Report any issues on the forums.
-
Tenboro
@ehentai
The wiki will be offline for a little bit while we finish migrating it to a new server.
-
Tenboro
@ehentai
New banners up. Thanks to Look A Moth, Pretty anon, Noeleo, nrdn25, DJNoni, sdpfsj and Grandmasters.
-
Tenboro
@ehentai
News banners updated. Thanks to sdpfsj, decondelite, cate_chan, Grandmasters, Look A Moth, nrdn25, DJNoni, Noeleo and Pretty anon.
-
Tenboro
@ehentai
News banners updated. Thanks to nasu_sensei, Look A Moth, Noeleo, DJNoni, Pretty anon, sdpfsj, TkzcM and cate_chan.
-
Tenboro
@ehentai
One of the uploaders had a network adapter malfunction which would have caused some issues for the last 20 minutes or so. It should now be back online.
-
Tenboro
@ehentai
News banners updated. Thanks to DJNoni, Look A Moth, Noeleo, nasu_sensei, sdpfsj, TkzcM and Spectre.
-
Tenboro
@ehentai
News banners updated. Thanks to TkzcM, sdpfsj, xxxhentaii, Shizuka Natsuki, Look A Moth, Noeleo, DJNoni and Titania1000.
-
Tenboro
@ehentai
We're seeing unusual packetloss on some internal routes which might be causing some sporadic failures. Host has been notified.
-
Tenboro
@ehentai
News banners updated. Thanks to DJNoni, TkzcM, Noeleo, Spectre, sdpfsj, Titania1000 and Look A Moth.
-
Tenboro
@ehentai
News banners updated. Thanks to TkzcM, xxxhentaii, ancientrule, Titania1000, sdpfsj, Look A Moth and DJNoni.
-
Tenboro
@ehentai
New banners up. Thanks to DJNoni, sdpfsj, Look A Moth, Flying Trillobit, TkzcM, Noeleo and WheldrakeX.
-
Tenboro
@ehentai
New banners are up. Thanks to Noeleo, sdpfsj, Look A Moth, TkzcM, WheldrakeX, DJNoni and F A E.
-
Tenboro
@ehentai
New news banners are up. Thanks to WheldrakeX, DJNoni, Look A Moth, Spectre, Noeleo, swimswim100 and sdpfsj.
-
Tenboro
@ehentai
Reminder: H@H clients still running 1.4.2 or earlier need to update to 1.6.0 by the end of this month to keep working.
-
Tenboro
@ehentai
New banners are up. Thanks to DJNoni, Noeleo, WheldrakeX, FruitfulFap, Look A Moth, sdpfsj, nasu_sensei, swimswim100 and lazyNPC.
-
Tenboro
@ehentai
Everything should now be back online. Let me know on the forums if something is broken.
-
Tenboro
@ehentai
We'll now start going offline for the maintenance.
-
Tenboro
@ehentai
Our main host has scheduled service-impacting maintenance tomorrow from 0500 to 0900 UTC, which means all services will be either down or in read-only mode. Due to the nature of the maintenance, we will start bringing things down after 0400 UTC, a bit more than 8 hours from now.
-
Tenboro
@ehentai
If you were running the experimental 1.5 build of H@H, you should revert to 1.4.2. 1.5 clients will soon stop receiving traffic.
-
Tenboro
@ehentai
New banners are up. Thanks to sdpfsj, FruitfulFap, Spectre, Noeleo, DJNoni, WheldrakeX and Look A Moth.
-
Tenboro
@ehentai
News banners updated. Thanks to FruitfulFap, Noeleo, sdpfsj, Flying Trillobit, Look A Moth, WheldrakeX and mayriad.
-
Tenboro
@ehentai
The connectivity issues appear to have been resolved.
-
Tenboro
@ehentai
Host is aware of the issues and working on resolving them.
-
Tenboro
@ehentai
One of our locations seem to be experiencing a connectivity outage, currently checking on this.
-
Tenboro
@ehentai
New news banners are up. Thanks to Look A Moth, Noeleo, WheldrakeX, Honeycat, TkzcM, FruitfulFap, sdpfsj and Tumbres.
-
Tenboro
@ehentai
News banners updated. Thanks to FruitfulFap, Tumbres, WheldrakeX, sdpfsj, Look A Moth, Honeycat, TkzcM and Noeleo.
-
Tenboro
@ehentai
The switchover was completed with no apparent issues. You may still see some old cache data here and there (especially with tags), but these should all be cleared up within 45 minutes or so. If you spot any problems, please report them on the forums.
-
Tenboro
@ehentai
Over the next hour or so, we will complete the switchover to a new web cluster. No downtime is expected, but you may see some minor (visual) temporary desyncs of recently changed galleries due to caching effects and different parts of the site using different clusters.
-
Tenboro
@ehentai
News banners updated. Thanks to WheldrakeX, Flying Trillobit, TkzcM, Honeycat, Look A Moth, Tumbres, Noeleo, sdpfsj and Srac.
-
Tenboro
@ehentai
News banners updated. Thanks to Honeycat, Noeleo, Look A Moth, Tumbres, Flying Trillobit, Srac, WheldrakeX, indiexanna, sdpfsj and Someguywith.
-
Tenboro
@ehentai
Traffic seems to be normalizing. One of our hosts reported a large fiber outage at Eurofiber at the same time Cloudflare broke, which may or may not be related. No official cause has been given yet.
-
Tenboro
@ehentai
It's funny because sites like DownDetector and IsItDownRightNow are down. Because they use Cloudflare, too.
-
Tenboro
@ehentai
Cloudflare is having issues again, which means you may be seeing 502 errors or non-loading pages.
-
Tenboro
@ehentai
The connectivity issues appear to have been resolved.
-
Tenboro
@ehentai
"We have identified a possible route leak impacting some Cloudflare IP ranges and are working with the network involved to resolve this." -Cloudflare
-
Tenboro
@ehentai
You may be seeing connectivity issues or general slowness on parts of the site due to global DNS/BGP issues affecting Cloudflare.
-
Tenboro
@ehentai
Banners updated. Thanks to WheldrakeX, Noeleo, Xythy, TkzcM, Srac, Look A Moth, Honeycat, sdpfsj, lazyNPC and Shiskk.
-
Tenboro
@ehentai
Updated news banners. Thanks to WheldrakeX, Xythy, Honeycat, Srac, Look A Moth, a545774001, Noeleo and lazyNPC.
-
Tenboro
@ehentai
News banners updated. Thanks to Look A Moth, Xythy, a545774001, Honeycat, Srac, WheldrakeX, lazyNPC and Noeleo.
-
Tenboro
@ehentai
The update has been applied. Please report any issues on the forums.
-
Tenboro
@ehentai
The process of applying the update will begin momentarily.
-
Tenboro
@ehentai
A major update to Galleries is scheduled to be pushed live tomorrow, March 19th, most likely some time between 0900 and 1000 GMT. The site will be in partial read-only mode while the update is applied, but uploads will not be affected.
-
Tenboro
@ehentai
The networking issues appear to have been resolved.
-
Tenboro
@ehentai
There is currently some sort of network outage affecting our primary location. The site will be in failover mode until this is resolved.
-
Tenboro
@ehentai
News banners updated. Thanks to Someguywith, Xythy, Paff444, Look A Moth, boulay, name83u5, Srac and Noeleo.
-
Tenboro
@ehentai
New banners are up. Thanks to Srac, Paff444, lazyNPC, thegodgame, Honeycat, Look A Moth, Someguywith and Xythy.
-
Tenboro
@ehentai
New news banners are up. Thanks to Xythy, lazyNPC, Paff444, Srac, Look A Moth, thegodgame, Five V and Honeycat. And a belated happy new year to everyone.
-
Tenboro
@ehentai
New news banners are up. Thanks to Xythy, Paff444, Srac, Look A Moth, lazyNPC, thegodgame and Honeycat.
-
Tenboro
@ehentai
New news banners are up. Thanks to Spectre, Look A Moth, gianfrix94, Noeleo, Paff444, Srac and Xythuard.
-
Tenboro
@ehentai
We had a brief outage caused by network problems at our main host. Failure mode seems to be the same as last time this happened.
-
Tenboro
@ehentai
We have received some reports that right-click saving no longer works in Firefox 63. This looks to be a bug in Firefox itself, so please use a different browser or the ESR version for now.
-
Tenboro
@ehentai
Our main location had a brief outage due to significant packet loss at our host's upstream provider. This appears to have been resolved already.
-
Tenboro
@ehentai
The host has reported that all maintenance was completed successfully.
-
Tenboro
@ehentai
Due to scheduled network maintenance, some site functionality might be degraded or unavailable during parts of the next few hours. Total expected downtime is excepted to be under 60 minutes.
-
Tenboro
@ehentai
News banners updated. Thanks to hzqr, Look A Moth, Srac, Xythuard, Noeleo, gianfrix94, Paff444, Sesshomaru Moon and unit111.
-
Tenboro
@ehentai
News banners updated. Thanks to Xythuard, Srac, Look A Moth, indiexanna, Paff444, Sesshomaru Moon, someguywith and unit111.
-
Tenboro
@ehentai
One of the upload servers experienced a hung network adapter, which may have caused some overnight issues. It has now been recovered.
-
Tenboro
@ehentai
New news banners are up. Thanks to someguywith, Shizuka Natsuki, Xythuard, Look A Moth, Paff444, Srac and Sesshomaru Moon.
-
Tenboro
@ehentai
News banners updated. Thanks to gianfrix94, Five V, Look A Moth, Srac, Xythuard, someguywith, jplshejeser, Paff444, and Sesshomaru Moon.
-
Tenboro
@ehentai
New news banners are up. Thanks to Srac, Look A Moth, Five V, Sesshomaru Moon, Xythuard, Paff444, gianfrix94, someguywith, Noeleo and savethequeen2.
-
Tenboro
@ehentai
New banners are up. Thanks to Look A Moth, Noeleo, SMiyano, Paff444, gianfrix94, Sesshomaru Moon, someguywith, Five V, Srac and Usagi =.
-
Tenboro
@ehentai
New banners are up. Thanks to Five V, Usagi =, Look A Moth, Srac, F A E, Noeleo, Paff444, gianfrix94 and Sesshomaru Moon.
-
Tenboro
@ehentai
New banners are up. Thanks to gianfrix94, Paff444, Look A Moth, Five V, Usagi =, Sesshomaru Moon, F A E and indiexanna.
-
Tenboro
@ehentai
Due to network maintenance at one of our hosts, we're seeing periods of high packet loss between some of our locations, which could cause degraded performance. This maintenance window started at 0100 UTC and is scheduled to end by 0600 UTC.
-
Tenboro
@ehentai
For people who use the H@H hourly bandwidth option: the ability to set this for schedules will soon be removed, and the hourly limit will also be averaged over the day rather than be strictly per hour. If you rely on this, make sure to set the throttle appropriately instead.
-
Tenboro
@ehentai
Assuming no last-minute issues, we'll start pushing an update with an overhaul to the profile/settings system over the next hour or so. At some point during this, your settings will reset. At this point, you can visit the settings page to have your old settings imported.
-
Tenboro
@ehentai
New banners are up. Grats to Anonymous, Sesshomaru Moon, Five V, Look A Moth, Usagi =, gianfrix94 and F A E.
-
Tenboro
@ehentai
The packetloss issues have been resolved.
-
Tenboro
@ehentai
We're currently seeing some fairly significant packetloss between two of our data centers, this may cause some slowness and failed operations.
-
Tenboro
@ehentai
New news banners are up. Grats to gianfrix94, Five V, zqjr, Sesshomaru Moon, F A E, Look A Moth and Usagi =.
-
Tenboro
@ehentai
Happy new year! あけおめ、ことよろ~
-
Tenboro
@ehentai
The frontend has been restored and appears to be operating normally.
-
Tenboro
@ehentai
One of our frontends has encountered a hardware failure, which will lead to reduced capacity and potentially other issues. It is currently being worked on.
-
Tenboro
@ehentai
The routing issues seem to have been resolved.
-
Tenboro
@ehentai
One of our hosts is currently experiencing some routing issues that is affecting availability.
-
Tenboro
@ehentai
New news banners are up. Grats to Five V, boulay, LOL50015, Zickma, F A E, clarkiest and hzqr.
-
Tenboro
@ehentai
New news banners are up. Grats to indiexanna, LOL50015, Zickma, ALL_MIGHT, eCode and F A E.
-
Tenboro
@ehentai
The network outage appears to have been resolved.
-
Tenboro
@ehentai
A partial outage at one of our hosts is currently causing issues connecting to some services.
-
Tenboro
@ehentai
One of our hosts has scheduled emergency network maintenance tomorrow, October 11th, between 0400 and 0600 UTC. This is expected to affect two of our frontends with network interruptions lasting 10-15 minutes.
-
Tenboro
@ehentai
Updated news banners are up. Grats to hzqr, indiexanna, LOL50015, ALL_MIGHT, eCode, Sesshomaru Moon and Mags_.
-
Tenboro
@ehentai
One of our hosts has scheduled core router upgrades on September 28th between 0200 and 0600 UTC. Some interruptions can be expected.
-
Tenboro
@ehentai
All of the servers have been restored, capacity is back to 100%.
-
Tenboro
@ehentai
One of our datacenters experienced a power outage, which may have caused minor issues. Slightly reduced capacity until servers are restored.
-
Tenboro
@ehentai
Updated news banners are up. Grats to LOL50015, boulay, Sesshomaru Moon, ALL_MIGHT, QR5, F A E and eCode.
-
Tenboro
@ehentai
The switch for the same frontend server seems to have failed again. Expect some failures as the traffic is rerouted.
-
Tenboro
@ehentai
HV is now back online.
-
Tenboro
@ehentai
Roughly one hour from now, HV will go down for an update. Battles should not reset.
-
Tenboro
@ehentai
The frontend has been restored. The problem seems to have been a faulty switch.
-
Tenboro
@ehentai
One of our frontends is having some issues, which could cause load failures. Currently investigating.
-
Tenboro
@ehentai
New news banners are up. Grats to Sesshomaru Moon, F A E, eCode, ALL_MIGHT, hzqr, LOL50015 and linardice.
-
Tenboro
@ehentai
The update caused a subtle malfunction on one resample image server, which led to a higher than usual image load failure rate. Fixed now.
-
Tenboro
@ehentai
The maintenance has been concluded. If you encounter any issues, please report it on the forums.
-
Tenboro
@ehentai
The maintenance will commence shortly. Estimated time required is 2-3 hours.
-
Tenboro
@ehentai
Galleries will be read-only for the full period. HV and Tracker will be down for parts of the period. Forum and Wiki will remain up.
-
Tenboro
@ehentai
We are planning some maintenance tomorrow around 1000 UTC, or roughly 12 hours from now. Expected duration is 2-3 hours.
-
Tenboro
@ehentai
The Google indexing issue should now be resolved.
-
Tenboro
@ehentai
And it's back.
-
Tenboro
@ehentai
HV needs some unscheduled maintenance. Should be back up in a bit.
-
Tenboro
@ehentai
New news banners are up. Grats to RaccCITY, yuripe, ALL_MIGHT, LOL50015, hzqr, F A E and eCode.
-
Tenboro
@ehentai
The wiki has been updated, let us know if you encounter any issues.
-
Tenboro
@ehentai
The wiki will be offline for a bit due to a major version update.
-
Tenboro
@ehentai
HV will go down in around 20 minutes to finish up some backend work. Expected downtime is around an hour.
-
Tenboro
@ehentai
HV will go down shortly for the 0.85 update. There will be some maintenance taking place during this time, so it could take an hour or so.
-
Tenboro
@ehentai
One of the uploaders experienced a hung network adapter which would have caused some uploads to fail or time out. It has now been recovered.
-
Tenboro
@ehentai
New news banners are up. Grats to RaccCITY, Lord Juice, blue penguin, Noeleo, lovepupils, gianfrix94 and eCode.
-
Tenboro
@ehentai
The network issues appear to have been resolved, performance is currently back to normal.
-
Tenboro
@ehentai
We're seeing some network routing issues at one of our hosts, which can lead to spotty performance and read-only moding.
-
Tenboro
@ehentai
New news banners are up. Grats to yuripe, lovepupils, gianfrix94, Lord Juice, blue penguin, n-nobody and Noeleo.
-
Tenboro
@ehentai
New news banners are up. Grats to gianfrix94, n-nobody, boulay, yuripe, Sanddan, Lord Juice and Scremaz.
-
Tenboro
@ehentai
New news banners are up. Grats to rraven314, Sesshomaru Moon, gianfrix94, Zetabolinho, Sanddan, Lord Juice and blue penguin.
-
Tenboro
@ehentai
The only function of the addon is flipping security.mixed_content.send_hsts_priming pref to false, so if you did that already, you're good.
-
Tenboro
@ehentai
Note that this only happens if you have the update toggle set to "Automatically install updates". These will not pop up a manual option.
-
Tenboro
@ehentai
The issues with Firefox 51 were fixed via a system addon update a few days ago. These download automatically and install at next restart.
-
Tenboro
@ehentai
New news banners are up. Grats to yuripe, Sesshomaru Moon, Frostbite, gianfrix94, xestrm and Sanddan.
-
Tenboro
@ehentai
Firefox 51 is currently broken, those options were not ready for stable and got enabled by mistake. bugzilla.mozilla.org/show_bug.cgi?id=1335134 @ExSquee
-
Tenboro
@ehentai
We're seeing some intermittent network issues with one of our hosts that could be causing availability issues.
-
Tenboro
@ehentai
Stick with the old version, or switch to a browser made by competent people who don't push known broken features to stable.
-
Tenboro
@ehentai
The new uploader interface has been deployed to live, and uploading functionality has been reenabled. Changes are listed on the forums.
-
Tenboro
@ehentai
Uploading will be turned off for maintenance and updates in a short while, expected downtime is less than an hour.
-
Tenboro
@ehentai
New banners are up. Grats to boulay, lovepupils, Sesshomaru Moon, xestrm, gianfrix94, Frostbite and Zetabolinho.
-
Tenboro
@ehentai
The routing issues were caused by a malfunctioning linecard, and should be resolved now.
-
Tenboro
@ehentai
You may be seeing some occasional failures caused by a routing issue with one of our hosts, particularly when generating new archives.
-
Tenboro
@ehentai
New banners are up! Grats to Japidura, Frostbite, Zetabolinho, Sesshomaru Moon, gianfrix94, xestrm and GET9.
-
Tenboro
@ehentai
New banners are up. Grats to GET9, boulay, xestrm, blue penguin, Sesshomaru Moon, Lord Juice and rraven314.
-
Tenboro
@ehentai
New banners are up. Grats to japidura, Sesshomaru Moon, Snittzi, yuripe, aiwotorimodose, indiexanna and Frostbite.
-
Tenboro
@ehentai
Connectivity appears to be restored. We're still seeing issues with IPv6 routing, but that only affects HV and should fall back to IPv4.
-
Tenboro
@ehentai
One of our hosts is currently experiencing a network outage. Galleries will be in read-only mode until it is resolved.
-
Tenboro
@ehentai
Galleries has now been switched back to the primary cluster. Everything should be fully operational, let us know if you spot any problems.
-
Tenboro
@ehentai
Power has been restored, and most services are back online. We are still preheating the main cluster caches before the gallery switchover.
-
Tenboro
@ehentai
We are now starting the process of shutting everything down for the power maintenance. See you on the other side.
-
Tenboro
@ehentai
The actual maintenance starts at 0800 UTC, and should last between six and eight hours. Galleries will still be available in read-only mode.
-
Tenboro
@ehentai
Reminder: When this tweet is 12 hours old, we'll start shutting down our primary rack for urgent power maintenance on the row it is located.
-
Tenboro
@ehentai
Apparently this was a revenge DDoS from a group that had one of their servers confiscated. This situation is now supposedly resolved.
-
Tenboro
@ehentai
One of our hosts has had degraded performance over the last 12 hours or so due to a large DDoS attack targeting their entire network.
-
Tenboro
@ehentai
Galleries will still be available in read-only mode, other services will generally be unavailable. Estimated downtime is 6-8 hours.
-
Tenboro
@ehentai
The upcoming Saturday, on the 24th at 10:00 CEST, we'll have some extended downtime due to "urgent" power maintenance at our primary host.
-
Tenboro
@ehentai
Connectivity appears to have been restored. The official cause was a faulty supervisor on a core router, and borked failover from that.
-
Tenboro
@ehentai
The connectivity for this host has gone from intermittent/degraded to down entirely, currently waiting for a status update.
-
Tenboro
@ehentai
We're seeing some intermittent connectivity failures at one of our hosts, which could degrade availability.
-
Tenboro
@ehentai
The network connectivity appears to have been restored.
-
Tenboro
@ehentai
The data center for one of our providers is currently experiencing an outage, which may affect availability.
-
Tenboro
@ehentai
A resample server was experiencing long RAID stalls, causing image loads to fail. Likely cause was a bad disk, which has now been replaced.
-
Tenboro
@ehentai
New banners are up. Grats to japidura, yuripe, boulay, aiwotorimodose and Sesshomaru Moon.
-
Tenboro
@ehentai
Corrected an issue with a cleanup script on the resampler that made it run out of scratch space for some operations.
-
Tenboro
@ehentai
Uploading has been re-enabled. Took slightly longer than expected due to a broken bootloader.
-
Tenboro
@ehentai
We'll be turning off gallery uploading in a sec for some quick updates.
-
Tenboro
@ehentai
The maintenance on the archivers has been completed.
-
Tenboro
@ehentai
Going to do some software upgrades on the archivers, individual archives could be unavailable for periods up to half an hour or so.
-
Tenboro
@ehentai
New banners are up. Grats to boulay, blue penguin, aiwotorimodose, Sesshomaru Moon, Lakituu and yuripe.
-
Tenboro
@ehentai
One of our virtual machine hosts decided to crash and reboot for no obvious reason, which could have caused unavailability. Back up now.
-
Tenboro
@ehentai
New banners are up. Grats to aiwotorimodose, Sesshomaru Moon, blue penguin, yuripe, lovepupils, Lord Juice and Lakituu.
-
Tenboro
@ehentai
It's been failed over to its redundant twin, but you may see some missing files while it catches up with the last few hours of resamples.
-
Tenboro
@ehentai
One of the resamplers seems to be experiencing some kind of storage system failure that's causing very high disk I/O delay.
-
Tenboro
@ehentai
A routing problem that would have caused issues accessing HV over IPv6 should now have been resolved.
-
Tenboro
@ehentai
New banners are up. Grats to Frostbite, boulay, lovepupils, Lakituu, aiwotorimodose, Lord Juice and yuripe.
-
Tenboro
@ehentai
One of our data centers is experiencing some partial intermittent connectivity issues that could affect availability.
-
Tenboro
@ehentai
New banners are up. Grats to boulay, lovepupils, aiwotorimodose, japidura, The Magi, Lakituu and yuripe.
-
Tenboro
@ehentai
Expect some brief downtime on some services about four hours from now due to maintenance on a core distribution router. (Don't Panic)
-
Tenboro
@ehentai
One of our data centers seems to be experiencing some intermittent connectivity issues. You may see occasional dropouts due to this.
-
Tenboro
@ehentai
Roughly one hour from now, HV will go down for an update and cluster transfer. Because of the move, ongoing battles WILL reset.
-
Tenboro
@ehentai
Chrome 50 appears to have a rendering bug that makes some text elements in HV fail to display properly. Use a different browser for now.
-
Tenboro
@ehentai
New banners are up. Grats to Sesshomaru Moon, The Magi, boulay, aiwotorimodose and yuripe.
-
Tenboro
@ehentai
HV is now back online.
-
Tenboro
@ehentai
Taking HV offline for a bit to fix a cluster failure.
-
Tenboro
@ehentai
One of the uploaders decided to drop its public-facing network adapter for no particular reason. Should be back online now.
-
Tenboro
@ehentai
The maintenance tasks have been successfully completed. All services are now back online.
-
Tenboro
@ehentai
The maintenance will start shortly.
-
Tenboro
@ehentai
Various other maintenance will also be done at this time, so other services will be unavailable for parts of the maintenance period.
-
Tenboro
@ehentai
Tomorrow, about 20 hours from now, we'll switch Galleries into read-only mode for 2-3 hours to replace the master database servers.
-
Tenboro
@ehentai
New banners are up. Grats to Sesshomaru Moon, The Magi, boulay, aiwotorimodose, FabulousCupcake, Zickma and yuripe.
-
Tenboro
@ehentai
Because of upstream emergency network maintenance, you may experience some failover mode and general slowness starting in a couple of hours.
-
Tenboro
@ehentai
Haven't received a proper RFO for last night's outage yet, outside of it being caused by a blackholing issue at our host's upstream.
-
Tenboro
@ehentai
Finishing up a round of server updates, you may see some brief interruptions while frontends are restarted.
-
Tenboro
@ehentai
One of our locations is having some intermittent network issues, so expect occasional failover mode until they get it fixed.
-
Tenboro
@ehentai
New banners are up. Grats to boulay, Sesshomaru Moon, japidura, aiwotorimodose, The Magi, Jackal Akabane and yuripe.
-
Tenboro
@ehentai
Most of the uploader functionality has been enabled. Note that you will see occasional broken thumbs until all the links are re-established.
-
Tenboro
@ehentai
Gallery uploads will be turned off in a little bit for the final changeover to new upload servers. Shouldn't take more than an hour.
-
Tenboro
@ehentai
The connectivity issues have all cleared up by now, all services should be back to normal.
-
Tenboro
@ehentai
A handful of servers are currently offline due to a distribution router failure. This can cause missing page elements on parts of the site.
-
Tenboro
@ehentai
Bounty and torrent uploads are available again. Poke us on the forums if you encounter any issues with the new servers.
-
Tenboro
@ehentai
Bounty and torrent uploads will be briefly unavailable while the functionality is moved to different servers.
-
Tenboro
@ehentai
While the URL domain highlighting should have made it obvious that these were bad links, please take proper precautions if you did click it.
-
Tenboro
@ehentai
A fair number of people seem to have received a phishing PM on the forums. These have now been mass deleted.
-
Tenboro
@ehentai
Because of some issues with live updating the logging servers, you may have been encountering some weirdness like duplicated dawns.
-
Tenboro
@ehentai
An error on a core database caused the site to enter read-only mode for a few hours. This should be clearing up now.
-
Tenboro
@ehentai
New banners are up. Grats to aiwotorimodose, saltwatermelon, Frostbite, Sesshomaru Moon, japidura, dtmhtk and boulay.
-
Tenboro
@ehentai
We had a brief ~10m network outage at our primary location. The site automatically entered failover mode, and should now be back to normal.
-
Tenboro
@ehentai
New banners are up. Grats to japidura, Sesshomaru Moon, yuripe, ChanForu, Frostbite, Dan31 and boulay.
-
Tenboro
@ehentai
The maintenance was successfully completed, all services have been re-enabled.
-
Tenboro
@ehentai
We'll be switching to read-only mode shortly to do the planned maintenance.
-
Tenboro
@ehentai
The uploaders are back online. Note that there will be a short read-only downtime tomorrow to correct another issue on a core server.
-
Tenboro
@ehentai
The uploaders will be briefly unavailable for some unscheduled maintenance.
-
Tenboro
@ehentai
An update caused some minor breakage on the upload servers, requiring a PHP module update/recompilation. Should be fixed now.
-
Tenboro
@ehentai
New banners are up. Grats to bahamutjr, The Magi, saltwatermelon, aiwotorimodose, Sesshomaru Moon, boulay and yuripe.
-
Tenboro
@ehentai
Reverted a broken update for the wiki. It should be available again.
-
Tenboro
@ehentai
New banners are up. Grats to yuripe, Sesshomaru Moon, The Magi, aiwotorimodose, jcullinane, Jackal Akabane and VTech100.
-
Tenboro
@ehentai
Doing another quick reboot of the forum server to fix a minor issue.
-
Tenboro
@ehentai
The forums are back online.
-
Tenboro
@ehentai
The forums will be briefly unavailable while we make some backend changes.
-
Tenboro
@ehentai
Events have been re-enabled.
-
Tenboro
@ehentai
Events (dawn + encounter) will be briefly unavailable due to maintenance.
-
Tenboro
@ehentai
New banners are up. Grats to boulay, khlaron, aiwotorimodose, phreeman, bahamutjr, gynew and japidura.
-
Tenboro
@ehentai
The wiki and two of the tracker nodes will be briefly unavailable while I restart a host virtual server.
-
Tenboro
@ehentai
We're getting some reports that Ghostery is broken with Firefox 40. If you're having issues with torrents, disable or remove it.
-
Tenboro
@ehentai
One of the image servers broke because an update silently turned something back on that really needed to be off. Should be fixed now.
-
Tenboro
@ehentai
The wiki should now available again as soon as the DNS has fully propagated. Let me know if you spot any problems.
-
Tenboro
@ehentai
The wiki will be unavailable for a bit while we move some things around.
-
Tenboro
@ehentai
Restarting a bunch of servers for updates. You may experience occasional unavailability for certain functions.
-
Tenboro
@ehentai
New banners are up. Grats to indiexanna, The Magi, aiwotorimodose, japidura, boulay, nintrun and khlaron.
-
Tenboro
@ehentai
In about half an hour, HV will be taken down for an update. Battles will not reset.
-
Tenboro
@ehentai
Applied some minor UI updates. Now using HTML5 placeholder for default input text. Torrent search reworked, can now also click user to list.
-
Tenboro
@ehentai
A new backup cluster has now been deployed. It has also been somewhat upgraded to back up certain services that the old one didn't.
-
Tenboro
@ehentai
The uploaders are now back online.
-
Tenboro
@ehentai
The uploaders will be briefly unavailable due to maintenance.
-
Tenboro
@ehentai
New banners are up. Grats to khlaron, saltwatermelon, Scarlet Flame, VTech100, The Magi, japidura and aiwotorimodose.
-
Tenboro
@ehentai
The control servers for H@H have been replaced. This should update automatically for everyone, unless you hardcoded the control IPs.
-
Tenboro
@ehentai
Corrected a server-side H@H issue that in rare cases could prevent a client from restarting if it wasn't correctly shut down.
-
Tenboro
@ehentai
Corrected a kernel limitation on the new archivers that could have caused slower speeds than normal for some visitors.
-
Tenboro
@ehentai
New banners are up. Grats to Frostbite, aiwotorimodose, saltwatermelon, khlaron, boulay, jcullinane and japidura.
-
Tenboro
@ehentai
Applied some updates to the core H@H handling to prevent some rare zombie client issues. Let me know if you spot any (unexpected) weirdness.
-
Tenboro
@ehentai
Because of this, the minimum compatible version of H@H required to connect was bumped up to 1.2.2. Older clients have to be updated.
-
Tenboro
@ehentai
The H@H settings screen was updated with the ability to remove some static ranges in order to reduce the max disk cache size.
-
Tenboro
@ehentai
On a related note, we have deployed six additional archivers with a total capacity of 1.2 PB per month.
-
Tenboro
@ehentai
Fixed a bug where some recent gallery archives could fail to download under certain circumstances.
-
Tenboro
@ehentai
Oh, and battles will reset, so make sure to wrap up whatever you are doing.
-
Tenboro
@ehentai
HV will go down around 0900 UTC for an update. It will take a while to run the database conversion, so expect at least an hour downtime.
-
Tenboro
@ehentai
New banners are up. Grats to Moenyaa, aiwotorimodose, indiexanna, japidura, Elvek, VTech100 and khlaron.
-
Tenboro
@ehentai
Fixed a rare bug where an archive upload could break if an hourly maintenance script ran at a very specific point as it was being processed.
-
Tenboro
@ehentai
Finished the maintenance on the wiki. Several major version updates, so let me know if you spot any weirdness.
-
Tenboro
@ehentai
The wiki will be down for maintenance, probably for up to an hour.
-
Tenboro
@ehentai
New banners are up. Grats to boulay, aiwotorimodose, yuripe, japidura, The Magi, saltwatermelon and jcullinane.
-
Tenboro
@ehentai
The data center is now back on utility power, we didn't experience any further power loss.
-
Tenboro
@ehentai
One of our primary data centers is currently on backup diesel generator power due to a power failure, likely the cause of the last event.
-
Tenboro
@ehentai
Two of the primary archiver/image servers went down due to a brief power outage to one of the server racks. They should now be back online.
-
Tenboro
@ehentai
The forum SSL certificate has been updated to stop Chrome from whining about weak SHA1 signatures.
-
Tenboro
@ehentai
A problem with the new tracker cluster where some torrents would show up as "not registered" should now be resolved.
-
Tenboro
@ehentai
EHTracker is currently migrating over to a new cluster. This could cause some fluctuating stats and less peers than usual for an hour or so.
-
Tenboro
@ehentai
The updates were successfully applied, and all services should again be available.
-
Tenboro
@ehentai
In about an hour, the site will be switched into read only to apply some updates to the main database. Downtime should be less than 30 min.
-
Tenboro
@ehentai
Also, note that some recent changes to the login system may be incompatible with certain third party addons. Might want to try not using it.
-
Tenboro
@ehentai
New banners are up. Grats to Moenyaa, Frostbite, aiwotorimodose, japidura, Wynaut, Bertaz and Scarlet Flame.
-
Tenboro
@ehentai
New banners are up. Grats to nano, VTech100, yuripe, Scarlet Flame, Frostbite, aiwotorimodose and jcullinane.
-
Tenboro
@ehentai
We've added a new set of thumbnail servers. Let me know if you encounter any issues with these.
-
Tenboro
@ehentai
New banners are up. Grats to Tiap, boulay, Moenyaa, Frostbite, Elvek, yuripe and Wynaut.
-
Tenboro
@ehentai
Happy holidays to everyone!
-
Tenboro
@ehentai
Going to restart a bunch of servers. You may see some brief connectivity interruptions as things reroute.
-
Tenboro
@ehentai
New banners are up. Grats to gynew, Sesshomaru Moon, Elvek, Scarlet Flame, Luminere, tiap and Frostbite.
-
Tenboro
@ehentai
As far as I can tell everything is in the clear, and the warnings should be clearing out eventually.
-
Tenboro
@ehentai
Apparently there had been some kind of intrusion in the ad network we use, but it was resolved by the time I started looking at it.
-
Tenboro
@ehentai
There seems to be some sneakiness in the ad code that gets flagged as suspicious, but the code samples provided are quite benign.
-
Tenboro
@ehentai
Some people are getting some kind of malware warning in Chrome, which has also been reported to us by Google. These are false positives.
-
Tenboro
@ehentai
New banners are up. Grats to Dan31, tiap, Wynaut, Tenshoku, Sesshomaru Moon, Frostbite and yuripe.
-
Tenboro
@ehentai
Yup, network is down again.
-
Tenboro
@ehentai
The network appears to be stable again. Still no RFO (Reason For Outage), host indicated one would be available within 24 hours.
-
Tenboro
@ehentai
One of our hosts is currently experiencing a network outage. We'll be in failover mode until that clears.
-
Tenboro
@ehentai
We had a partial outage due to a large DDoS against one of our hosts (not targeting us). Everything should be back to normal by now.
-
Tenboro
@ehentai
The trackers are back online.
-
Tenboro
@ehentai
The torrent trackers will be briefly unavailable due to maintenance.
-
Tenboro
@ehentai
New banners are up. Grats to gynew, tiap, aiwotorimodose, indiexanna, jcullinane, Wynaut and boulay.
-
Tenboro
@ehentai
HV will be briefly unavailable for an update.
-
Tenboro
@ehentai
The archiver maintenance has been completed.
-
Tenboro
@ehentai
Doing some maintenance on one of the archivers. Some archives will be unavailable for up to an hour.
-
Tenboro
@ehentai
New banners are up. Grats to Luminere, Zetta-byte, FATAL1T3, yuripe and Kagoromo.
-
Tenboro
@ehentai
After some further work by our host, it does seem like the issues we were experiencing have been fixed. No issues the last 24 hours or so.
-
Tenboro
@ehentai
The maintenance appears to have done nothing to fix the persistent packet loss issues. At this point I'm exploring other options.
-
Tenboro
@ehentai
Our host has scheduled an emergency maintenance for tonight at 2100 UTC in order to resolve the intermittent packet loss issues.
-
Tenboro
@ehentai
The routing problems appear to be resolved.
-
Tenboro
@ehentai
We're currently experiencing a full outage of our main cluster seemingly due to a major routing/BGP failure.
-
Tenboro
@ehentai
The backup cluster is back online.
-
Tenboro
@ehentai
The backup cluster is currently offline due to power maintenance on the rack. It will be back up as soon as maintenance completes.
-
Tenboro
@ehentai
New banners up. Grats to FATAL1T3, flint, aiwotorimodose, d3n, Shiro Akisu, yuripe and Luminere.
-
Tenboro
@ehentai
One of the frontends decided to take a break. It's back up and running now.
-
Tenboro
@ehentai
The forum is back online, and the replica has been rebuilt. Going to do the wiki as well while I'm at it.
-
Tenboro
@ehentai
The forums will be briefly unavailable while I apply some updates and rebuild a broken replica.
-
Tenboro
@ehentai
New banners are up. Grats to Dreamophobia, tiap, Fatal1t3, Greatho, Trombe1, aiwotorimodose and Luminere.
-
Tenboro
@ehentai
HV will be shortly be temporarily unavailable due to an update.
-
Tenboro
@ehentai
New banners are up. Grats to indiexanna, hihohahi, aiwotorimodose, Tenseigamoon, FALAT1T3, tiap and DeadlyFatalis.
-
Tenboro
@ehentai
There appears to be some widespread issues on backbone connections between certain providers, if the site is slow for you this could be why.
-
Tenboro
@ehentai
It only took nine years, but the forum login will now let you recover your username based on your email address. Progress™
-
Tenboro
@ehentai
New banners are up. Grats to hihohahi, Frostbite, rainbowmagnum, Xtrao, indiexanna, yuripe and katsugawa.
-
Tenboro
@ehentai
The secondary cluster will be busy applying the updates for the next 2-3 hours, so we'll be operating without the failover until then.
-
Tenboro
@ehentai
Maintenance has been completed, everything is now back up and running from the primary web cluster.
-
Tenboro
@ehentai
The site will be entering read only mode for maintenance in about 20 minutes. Upload functionality will be disabled shortly.
-
Tenboro
@ehentai
Tomorrow, May 1st at around 0800 UTC, the site will enter read-only mode for a couple of hours for database maintenance. HV will be down.
-
Tenboro
@ehentai
One of our frontends/thumbnail servers went down for no discernible reason. It should be back online now.
-
Tenboro
@ehentai
An unpatched 0-day exploit is circulating for all versions of Internet Explorer. If you didn't switch browser yet, no time like the present.
-
Tenboro
@ehentai
The packetloss some people are experiencing is caused by a transatlantic fiber cut. Our host's upstream is currently working on it.
-
Tenboro
@ehentai
Some regions seem to be suffering from high packet loss right now. Can't say exactly where the problem is, but it's not on our end.
-
Tenboro
@ehentai
The tracker will be briefly unavailable due to maintenance. And rainbows.
-
Tenboro
@ehentai
The forums will be briefly unavailable due to maintenance.
-
Tenboro
@ehentai
This isn't the first time, so we've now gone ahead and deployed our own DNS servers. It may however take a while before this propagates.
-
Tenboro
@ehentai
Because of some DNS issues at our domain host, you may be encountering some issues with some static content like thumbnails.
-
Tenboro
@ehentai
New banners are up. Grats to darkx, Tenshoku, Kagoromo, Dreamophobia, Kiorika, Frostbite and Scarlet Flame.
-
Tenboro
@ehentai
Fixed an issue with the H@H C&C that could have made clients reject responses from the server. (Especially during startup.)
-
Tenboro
@ehentai
One of our hosts had a full network outage overnight. This would have caused missing functionality, as the site ran from the backup cluster.
-
Tenboro
@ehentai
Some network issues affecting one of our frontends could have caused some slowness/drops for a while. Seems to have cleared up.
-
Tenboro
@ehentai
New banners are up. Grats to ctxl, Hassirith, katsugawa, Dreamophobia and Arthur Leibowitz.
-
Tenboro
@ehentai
The wiki was updated to the most recent stable version. Also fixed an issue where outdated pages could be displayed when not logged in.
-
Tenboro
@ehentai
The wiki will be briefly unavailable for a version update.
-
Tenboro
@ehentai
Assuming everything works out, thanks to these we can finally offer large thumbnails to all our registered visitors. (Already enabled.)
-
Tenboro
@ehentai
We just (partially) deployed a couple of new frontends. These both have 10 Gbps connectivity as well as ~3 TB of solid state storage.
-
Tenboro
@ehentai
Fixed an issue where a couple of frontends had cached some broken CSS, which led to various layout problems for some visitors.
-
Tenboro
@ehentai
The site has automatically entered failover mode due to a network outage, probably a large DDoS against our host. Haven't heard back yet.
-
Tenboro
@ehentai
Because stand-alone medium thumbnails are being retired, some (very new or private) galleries will now only have large thumbnails available.
-
Tenboro
@ehentai
The generation of new combine thumbnails started waay back on December 22nd has finally finished, about four days off the estimate.
-
Tenboro
@ehentai
New banners are up. Grats to radixius, FLCLocust, hihohahi, katsugawa, Frostbite, MrJoeX and yuripe.
-
Tenboro
@ehentai
The Multi-Page Viewer should now work properly with W3C-compatible touch interfaces (Chrome/Safari/Firefox).
-
Tenboro
@ehentai
This is both more secure, and also fixes a problem for people with certain characters in the password. Post about any issues on the forums.
-
Tenboro
@ehentai
The log-in mechanism when you log on from the front page or galleries has been changed, and now integrates directly with the forum log-in.
-
Tenboro
@ehentai
Lofi has now been switched over to use the new image/archive servers.
-
Tenboro
@ehentai
The network problems appear to be resolved. Failover mode seemed to work great for the most part, but still some shortcomings to address.
-
Tenboro
@ehentai
We're currently experiencing some network issues. On the bright side, it's the perfect chance to test the new failover read only mode!
-
Tenboro
@ehentai
The forum database has been successfully migrated. Total downtime was about 20 minutes.
-
Tenboro
@ehentai
In about an hour, I'll be taking the forum offline to migrate its database to a different server. Nothing else will be affected by the move.
-
Tenboro
@ehentai
The wiki has been successfully migrated, and most people should see the new DNS already.
-
Tenboro
@ehentai
The wiki will be briefly unavailable while it is migrated to a new server. Should only take ten minutes or so.
-
Tenboro
@ehentai
The network outage appears to have been resolved.
-
Tenboro
@ehentai
One of our frontends is currently offline due to a core network hardware failure. It's currently rerouting, which can take up to an hour.
-
Tenboro
@ehentai
Two new archiver servers have now gone live. These both have unmetered gigabit connectivity, and will eventually replace the existing ones.
-
Tenboro
@ehentai
Incidentally, HTTPS on the forums now supports Perfect Forward Secrecy with newer browsers. (Chrome 27+, Firefox 21+, IE9+, Safari 5.1+)
-
Tenboro
@ehentai
The frontend was successfully migrated. There will be a slightly longer downtime in the not too distant future to migrate the database.
-
Tenboro
@ehentai
In a short while, we'll be taking the forums briefly offline to switch the frontend to a new server. Downtime should only be a few minutes.
-
Tenboro
@ehentai
New banners are up. Grats to Kagoromo, aiwotorimodose, katsugawa, ctxl, Xtrao, Frostbite and Tenseigamoon.
-
Tenboro
@ehentai
Seeing as half of our visitors have seen the new year while the other half haven't, we wish you all a premature/belated Happy New Year.
-
Tenboro
@ehentai
Fixed an issue with archives where it failed to ensure unique filenames (and therefore couldn't generate the archive) in some specific cases
-
Tenboro
@ehentai
Added a toggle under My Settings to make the Archiver auto-accept and/or auto-start downloads.
-
Tenboro
@ehentai
The wiki will be briefly unavailable for some updates.
-
Tenboro
@ehentai
Due to some caching issues, some older galleries were attempting to use the new tilesets prematurely. This should now be fixed.
-
Tenboro
@ehentai
This process will take about 40 days, but newer galleries are prioritized, and they start using the new tilesets as they become available.
-
Tenboro
@ehentai
With the new uploaders, we have started a process of regenerating thumbnail tilesets from source, using a new and higher quality technique.
-
Tenboro
@ehentai
The old trackers have now been turned off. If you're not seeing the new ones yet, you should check for manually overridden DNS pointers.
-
Tenboro
@ehentai
We're currently switching in some new torrent trackers (technically, the old uploaders). You may see fewer peers while it propagates.
-
Tenboro
@ehentai
HV will be down briefly for an update.
-
Tenboro
@ehentai
Pushed a smallish update to Galleries. Mostly bugfixes, but it also adds a thumbnail mode for the favorite screen.
-
Tenboro
@ehentai
HV is now back online.
-
Tenboro
@ehentai
Taking HV offline for a while to fix some issues with Cassandra.
-
Tenboro
@ehentai
We had a brief period of unavailability due to a network outage at one of our hosts. Seems to be resolved by now.
-
Tenboro
@ehentai
The uploading mechanism has now been fully migrated to the new servers. If you encounter any issues, please let us know on the forums.
-
Tenboro
@ehentai
The second new uploader has now replaced the remaining older one as the active syncing partner. It will need some time to fully catch up.l
-
Tenboro
@ehentai
New banners are up. Grats to Trombe1, Annsomni, ctxl, Tenseigamoon, Arthur Leibowitz, Yuripe and Kagoromo.
-
Tenboro
@ehentai
We'll leave it like this until tomorrow, to make sure everything is working as it should.
-
Tenboro
@ehentai
One of the two old uploader servers have now been replaced with a new one, which is actively syncing with the remaining older server.
-
Tenboro
@ehentai
I've started the process of migrating the upload servers. In about 1 hour, one of the new ones will start replacing one of the old ones.
-
Tenboro
@ehentai
Because of the built-in redundancy we have on the uploading mechanism, no actual downtime is expected.
-
Tenboro
@ehentai
This weekend we will most likely be migrating to a new set of upload servers that have been under testing for the last month.
-
Tenboro
@ehentai
As the deadline passed, about 7000 monsters were forced into early retirement. You'll have to make due with the 42086 remaining ones.
-
Tenboro
@ehentai
Final heads-up: In less than four days, monsters that haven't been fed or upgraded for 180 days will start to be deactivated and un-named.
-
Tenboro
@ehentai
New banners are up. Grats to Rarfex, Annomni, Kagoromo, yuripe, aiwotorimodose, Arthur Leibowitz and FLCLocust.
-
Tenboro
@ehentai
Just had three servers go down due to a power failure at one of our hosts. (Curiously, not the same as last time). They are now back online.
-
Tenboro
@ehentai
We're currently doing a limited live test of a new H@H distribution mechanism. (A beta H@H build that uses this will be available soon-ish.)
-
Tenboro
@ehentai
This should help prevent actual breakage if Cassandra decides to fragment in the future.
-
Tenboro
@ehentai
Some of the backend nodes on the primary web cluster were renumbered to put them on the same VLAN. (Originally they were split between two.)
-
Tenboro
@ehentai
PSA: If you don't like your account, installing random Chrome plugins that let people proxy off your connection is an excellent idea.
-
Tenboro
@ehentai
The image server finished rebuilding its RAID, and is now back online.
-
Tenboro
@ehentai
One of the image servers lost a disk on its RAID. (Yes, this happens a lot.) It's already rebuilding on a new disk.
-
Tenboro
@ehentai
The thumbnail server has been rebuilt and is back online.
-
Tenboro
@ehentai
Either way, to fix it I'll have to take it offline and rebuild it with a higher count. If anything is missing thumbs, this is why.
-
Tenboro
@ehentai
One of the thumbnail servers seem to have gone and run out of inodes on its storage device. 61 million ought to be enough for everyone.
-
Tenboro
@ehentai
HV is back online.
-
Tenboro
@ehentai
Welp, the Cassandra cluster broke apart again. HV will be down for a while.
-
Tenboro
@ehentai
New banners are up. Grats to Trombe1, aiwotorimodose, Frostbite, rainbowmagnum, Tenseigamoon, mechafujoshi and hujan86.
-
Tenboro
@ehentai
This will start happening around the middle of November. Note that deactivated monsters will be un-named, but not deleted.
-
Tenboro
@ehentai
As a reminder, the first monsters below PL 250 that haven't been fed for more than six months are nearing the point where they deactivate.
-
Tenboro
@ehentai
The old, previously disabled Power Tank trainings have now been removed and refunded.
-
Tenboro
@ehentai
Note that uploads will likely be slower than usual, until the caches warm up and the deluge of uploads taper off.
-
Tenboro
@ehentai
All servers are now back online and all services should be restored. Though I will be running some background verification on the uploaders.
-
Tenboro
@ehentai
HV is back online. We're still waiting for one more server to come back up after the power outage to restore the last services.
-
Tenboro
@ehentai
There are two servers that aren't responsive for one reason or another, possibly that they weren't actually on A+B power. Investigating.
-
Tenboro
@ehentai
We had an hour-long outage due to a power outage at our primary facility. While our rack was not affected, apparently the network gear was.
-
Tenboro
@ehentai
HV will be down until I can figure out why.
-
Tenboro
@ehentai
For some baffling reason, a single (non-seed) node failure combined with a network outage caused Cassandra to fragment into three clusters.
-
Tenboro
@ehentai
We had a short outage due to some network issues, specifically BGP route flapping on our provider's uplinks. Seems to be back to normal.
-
Tenboro
@ehentai
Wiki should be back online now.
-
Tenboro
@ehentai
The wiki is currently offline due to a SAN failure on the so-called cloud it's hosted on.
-
Tenboro
@ehentai
Completed the repository verification, a single corrupt file was restored from another server. It is now back online.
-
Tenboro
@ehentai
RAID issues have been resolved, but I'll be running a full repository verification before putting it back online. Which will take ~3 days.
-
Tenboro
@ehentai
One of the (redundant) image servers had a multiple drive failure on its RAID. Going to take it offline for repairs.
-
Tenboro
@ehentai
HV will go down in about half an hour for an update. All active battles WILL reset.
-
Tenboro
@ehentai
Network issues appear to be resolved. I'll be restoring the site to the primary cluster over the next half hour or so.
-
Tenboro
@ehentai
Note that some functionality will be missing, behaving weirdly, or be outright disabled (like HV), as the system isn't ready for this setup.
-
Tenboro
@ehentai
Because the router issues have returned, I've temporarily failed the site over to the experimental secondary cluster for the duration.
-
Tenboro
@ehentai
New banners are up. Grats to katsugawa, Frostbite, Izur, rainbowmagnum, KuroSawaShirou, aiwotorimodose and Rarfex.
-
Tenboro
@ehentai
There were some more interruptions caused by the same router. Heard back from the DC that the route has been disabled entirely for now.
-
Tenboro
@ehentai
The problems with the router appear to have been resolved. Still waiting for an exact RFO.
-
Tenboro
@ehentai
Router problems are still ongoing. At this point I'm working on rush finalizing the deployment of a second cluster that's been in testing.
-
Tenboro
@ehentai
We're currently having some issues due to a router malfunction at one of our hosts. Hopefully they'll have it resolved in not too long.
-
Tenboro
@ehentai
Fixed an issue on the forums that may have prevented the "new user" captchas from showing for users with Firefox 23.
-
Tenboro
@ehentai
We're experiencing some intermittent network outages due to problems with upstream providers used by both our primary hosts.
-
Tenboro
@ehentai
New banners are up. Grats to Frostbite, Canavi, KuroSawaShirou, Trombe1, katsugawa, yuripe and aiwotorimodose.
-
Tenboro
@ehentai
We're currently having some issues due to a near-total network outage at one of our hosts. Hopefully they'll have it back online soon.
-
Tenboro
@ehentai
A couple frontend nodes were having some moderate network issues, these appear to be resolved now.
-
Tenboro
@ehentai
Fixed a rare replication timing issue where galleries could temporarily end up in a weird state where they could not be published.
-
Tenboro
@ehentai
The maintenance was completed, and everything should be back online.
-
Tenboro
@ehentai
The maintenance period will begin in about 10 minutes. Gallery uploads have been disabled.
-
Tenboro
@ehentai
Tomorrow around 10:00 UTC there will be another maintenance period to finalize a database major version update. Est about 1-2 hour downtime.
-
Tenboro
@ehentai
New banners are up. Grats to Canavi, DeadlyFatalis, aiwotorimodose, thegodgame, Tenseigamoon, indiexanna and Frostbite.
-
Tenboro
@ehentai
Fixed a caching issue for low-traffic galleries introduced by the recent maintenance.
-
Tenboro
@ehentai
The maintenance was successfully completed with plenty of time to spare. If you encounter any issues, please report them on the forums.
-
Tenboro
@ehentai
We will be going offline momentarily for the previously announced maintenance period.
-
Tenboro
@ehentai
The maintenance period will start in an hour and change. Uploading will be disabled roughly half an hour before.
-
Tenboro
@ehentai
Tomorrow morning, around 10:00 UTC, there will be a roughly one hour site-wide maintenance period to make some core changes to the database.
-
Tenboro
@ehentai
This does not require a client upgrade, but if you suspect that yours is measured as slower than it should be, you may want to restart it.
-
Tenboro
@ehentai
H@H now uses an improved, multi-threaded approach for performing the initial speed test. Should be particularly noticeable for fast clients.
-
Tenboro
@ehentai
Fixed an issue where the gallery stat tracking was partially dropping stats due to running out of temporary storage.
-
Tenboro
@ehentai
Seems like the issues were resolved fairly quickly, DNS seems to be back up and running.
-
Tenboro
@ehentai
We seem to be having some issues due to a major outage at GoDaddy. Don't really know the exact cause.
-
Tenboro
@ehentai
This shouldn't cause any problems, but it will increase the background load by quite a bit.
-
Tenboro
@ehentai
Over the next week, there will be a major database reindexing task running in the background, which changes how files are tied to galleries.
-
Tenboro
@ehentai
Fixed an issue where the full EHTracker processing load was put on just one of the two servers. Through it actually handled it pretty well.
-
Tenboro
@ehentai
Lost a disk on an image/archive server, which was replaced and is now rebuilding. These events barely qualify as news at this point.
-
Tenboro
@ehentai
The wiki will be briefly unavailable due to a VM node upgrade.
-
Tenboro
@ehentai
Applied some further optimizations to the tracker. These improve database caching for "negative hits", that is, for non-tracked torrents.
-
Tenboro
@ehentai
Note that if you have any old saved download links laying around, they will be invalidated, so you'll have to get new ones.
-
Tenboro
@ehentai
Currently applying some performance and reliability improvements to the archivers. These make them fully independent of the master database.
-
Tenboro
@ehentai
Put in some optimizations to the tracker and fully reworked the stat gathering pipeline. Appears to have fixed some issues with load spikes.
-
Tenboro
@ehentai
Updated the Twitter integration to use the 1.1 API, since they went and shut down the 1.0 one. Let's see if this works, shall we.
-
Tenboro
@ehentai
About to apply a new update to Galleries. HV will also be unavailable for the duration.
-
Tenboro
@ehentai
The routing problems appear to be resolved.
-
Tenboro
@ehentai
We're currently having some issues due to routing failures at Fiberring. Currently waiting for our providers to reroute traffic.
-
Tenboro
@ehentai
The wiki will be unavailable for about half an hour for a server migration.
-
Tenboro
@ehentai
New banners are up. Grats to Canavi, Tenseigamoon, Handsome Phil, alfanders, Ichy, hujan86 and yuripe.
-
Tenboro
@ehentai
The RAID was rebuilt and the server is back online.
-
Tenboro
@ehentai
One of the frontend/thumbnail servers has a bad disk, again. It's on a hot-swappable RAID, but I'll still take it offline to let it rebuild.
-
Tenboro
@ehentai
One of the image/archiver servers crashed overnight. It is back online now.
-
Tenboro
@ehentai
Restarting some more servers. This should have minimal effect, but may result in some missing thumbnails and failed tracker announces.
-
Tenboro
@ehentai
The forums will be briefly unavailable due to a server update/restart.
-
Tenboro
@ehentai
HV will go down for an update in an hour or so. All battles will reset, and downtime could be as much as a couple of hours.
-
Tenboro
@ehentai
Most servers will be restarted in batches to apply a kernel security update. Various parts of the site will be periodically unavailable.
-
Tenboro
@ehentai
We had an apparent parasprite infestation in one of our network routes. Should be rerouted and back to 100% now.
-
Tenboro
@ehentai
The karma-related hath perks have now been removed and refunded.
-
Tenboro
@ehentai
New banners are up. Grats to Frostbite, Shuleria, ctxl, mrwayne, aiwotorimodose, nuruponurupo and vash5e7.
-
Tenboro
@ehentai
The old retired Refined Aura and Karma Amplifier trainings have now been refunded.
-
Tenboro
@ehentai
For two weeks, nothing happened. But there was a large DDoS against one of our hosts a few hours ago which caused some minor disruptance.
-
Tenboro
@ehentai
Everyone who participated in the 2013 Easter Event should now have received their rewards. Enjoy~
-
Tenboro
@ehentai
Note that the next version of HV will be "always on". That's the world we live in. #dealwithit
-
Tenboro
@ehentai
New banners are up. Grats to Shuleria, Tenseigamoon, ctxl, mrwayne, roxxxas, unknownxz and Lord Smooze. Uh, Juice.
-
Tenboro
@ehentai
As the Snowflake event started later than normal, I've extended it until April 4th 23:59:59, so you now still have a chance to get your set.
-
Tenboro
@ehentai
(And don't believe everything you read on the interwebs, especially not on April 1st.)
-
Tenboro
@ehentai
HV easter event ends in about three hours, so if you didn't get your snowflakes in order, now's the time.
-
Tenboro
@ehentai
Note that due to a change in database format, the next HV patch will feature an equipment wipe. MBP Tails will be handed out to compensate.
-
Tenboro
@ehentai
The site was apparently having some issues around midnight UTC due to a network outage. Seems like it recovered by itself.
-
Tenboro
@ehentai
Tomorrow, Friday starting at 09:00 UTC, some site functions will be unavailable due to a datacenter/server migration at one of our hosts.
-
Tenboro
@ehentai
About to apply a core update to some functionality that touches on all parts of the site. No downtime is expected, if all goes as planned.
-
Tenboro
@ehentai
The disk was replaced, and the RAID is now rebuilding.
-
Tenboro
@ehentai
Seeing a bad disk on one of the image servers. It's in a RAID, so it should remain live during replacement and rebuild.
-
Tenboro
@ehentai
Added some further improvements to H@H to improve the handling of clients that repeatedly reports failures when testing other clients.
-
Tenboro
@ehentai
This to prevent it from being blocked by default by certain browsers.
-
Tenboro
@ehentai
If your H@H client is running on one of the following ports, you should change it: 2049,3659,4045,6000,6665,6666,6667,6668,6669,65535
-
Tenboro
@ehentai
H@H now uses a new "quality" metric that measures a client's long-term stability, speed and reliability to prioritize file requests.
-
Tenboro
@ehentai
The tweaks to the cluster nodes seem to have done the trick, there have been no more errors logged for the distributed memory object cache.
-
Tenboro
@ehentai
Going to restart some of the web cluster nodes to apply some updates and configuration tweaks. Could cause some very short disruptions.
-
Tenboro
@ehentai
HV will be down briefly for an update.
-
Tenboro
@ehentai
New banners are up. Grats to Death Grunty, Handsome Phil, DeadlyFatalis, ShatteredWings, Deep Snow, hujan86 and hentai_inu.
-
Tenboro
@ehentai
Added some fixes to the uploader to prevent it from balking on certain files, and to properly handle archives containing Japanese filenames.
-
Tenboro
@ehentai
Disruption was caused by a router blade crash at one of our host's upstreams. It's been stable the last hour and change.
-
Tenboro
@ehentai
Currently seeing some more periodic network disruptions, which could be causing issues.
-
Tenboro
@ehentai
Hopefully that's the end of that...
-
Tenboro
@ehentai
Connectivity is back to being awful. Some kind of large DDoS going on in our neighborhood.
-
Tenboro
@ehentai
Seems to have cleared up now.
-
Tenboro
@ehentai
Currently seeing quite a bit of packet loss, which is probably making stuff perform like crap.
-
Tenboro
@ehentai
HV will be down shortly for an update.
-
Tenboro
@ehentai
Updated the ruleset on the reverse proxies to avoid mistakenly caching pages with certain URLs that matched a filter for static content.
-
Tenboro
@ehentai
Fixed a problem with the self-monitoring system where it might store an incomplete state that, if your timing sucked, could make stuff fail.
-
Tenboro
@ehentai
The archiver should be back online now.
-
Tenboro
@ehentai
.. which should resolve some issues with generating new archivers caused by the downed one.
-
Tenboro
@ehentai
Added some improvements to the remaining archivers. They now use the site's internal self-monitoring system to prevent hitting dead servers.
-
Tenboro
@ehentai
Doesn't seem like it wants to come back online. Will have to ask host to have a look-see.
-
Tenboro
@ehentai
One of the archiver/image servers seems to have encountered some kind of hardware hickup that caused a CPU soft lockup. Going to restart it.
-
Tenboro
@ehentai
We're seeing some packet loss on one of our internal routes, which is causing some load failures. Host is looking into it.
-
Tenboro
@ehentai
The patch was applied without any issues. Patch notes as usual: forums.e-hentai.org/index.php?showtopic=114137
-
Tenboro
@ehentai
About to push an update to Galleries. No downtime is expected.
-
Tenboro
@ehentai
Fixed a problem with some frontends where some configuration tweaks were overwritten by an update, which was causing some load fails.
-
Tenboro
@ehentai
(Well, it's not strictly "new", as it's been thoroughly tested in a replica role for about eight months.)
-
Tenboro
@ehentai
At first glance, the main database seems a lot happier on the new Dual E5-2620 with 64 GB RAM than on the old X3430 with 16 GB.
-
Tenboro
@ehentai
And we're back. Did you miss us? (Don't answer that.) Let me know if anything is broken.
-
Tenboro
@ehentai
The migration and server move have both been completed. The system is currently reinitializing and should be live in ten minutes or so.
-
Tenboro
@ehentai
Everything is now offline, and the move and database migration are both underway.
-
Tenboro
@ehentai
Reminder: in less than an hour, the site will go offline for a couple of hours due to a combined server move and main database changeover.
-
Tenboro
@ehentai
Finished this round of upgrades without any major issues.
-
Tenboro
@ehentai
In preparation for tomorrow's downtime, I'm going to do a rolling upgrade on the cluster nodes. Might cause some very short interruptions.
-
Tenboro
@ehentai
Note that the site will be fully offline during this period. Estimated downtime is about two hours.
-
Tenboro
@ehentai
Tomorrow at around 0830 UTC, we'll be performing the last of our server moves. At the same time, the main database will be switched.
-
Tenboro
@ehentai
Opera ditches own rendering engine, switches to WebKit (Safari/Chrome). How I wish the first word of that sentence was "Microsoft".
-
Tenboro
@ehentai
Peak effective Hentai@Home capacity now exceeds 10 Gbit/s. Thanks to everyone who keeps those files flowing.
-
Tenboro
@ehentai
New banners are up. Grats to Death Grunty, brzonogizmaj, Dan31, jculliane, DeadlyFatalis, aiwotorimodose and ctxl.
-
Tenboro
@ehentai
One of the image/archive servers decided to take a dive. It should be back online now.
-
Tenboro
@ehentai
The forum will now analyze all posted URLs in posts and PMs, and explicitly display the actual target domain before the URL itself.
-
Tenboro
@ehentai
Fixed a problem on gallery comments that could make the system reject them with a "nope" message under rare circumstances.
-
Tenboro
@ehentai
Oh, and the move was completed without any apparent issues.
-
Tenboro
@ehentai
State tracking is being moved from its temporary host to its permanent location. HV will be down briefly while this happens.
-
Tenboro
@ehentai
Fixed a problem with caching on the Wiki that led to unspecified behavior when accessing pages in the File: namespace.
-
Tenboro
@ehentai
The forum has been moved. Please report any issues if you spot anything.
-
Tenboro
@ehentai
In a little while, I'll be migrating the forums to a different server. Expected downtime for the forum is about 1/2 hour, mostly for DNS.
-
Tenboro
@ehentai
Made some tweaks to the thumbnail load distribution to take the age (and presumably load) of galleries into account.
-
Tenboro
@ehentai
HV will be down briefly for an update.
-
Tenboro
@ehentai
All the thumbnail servers have been updated to the 1.2 branch of nginx. Among other things, this re-enables keepalive for Safari users.
-
Tenboro
@ehentai
The server was successfully moved and is back online.
-
Tenboro
@ehentai
Another server is being shut down in order to move it into the new rack. Shouldn't affect the live site in any way.
-
Tenboro
@ehentai
Fixed a problem with one of the image servers that prevented it from retrieving new images.
-
Tenboro
@ehentai
Fixed a complex problem with the H@H C&C that would in rare cases lead to clients being suspended for failing to check in even if they had.
-
Tenboro
@ehentai
The wiki has been migrated. As soon as the DNS has propagated, you should see the new version. (Not that it looks any different.)
-
Tenboro
@ehentai
EHWiki will be unavailable for about half an hour while I migrate it to another server and update it to the latest version.
-
Tenboro
@ehentai
The state tracking has been switched over successfully. Can't see any issues, but report them if you see them.
-
Tenboro
@ehentai
About to switch over logging/state tracking to other servers. HV will be briefly unavailable in little while to copy player states.
-
Tenboro
@ehentai
The forum will now send a warning to your original email address whenever the password or email associated with the account is changed.
-
Tenboro
@ehentai
Corrected some configuration issues with the new tracker that made it reject particularly verbose clients.
-
Tenboro
@ehentai
EHTracker is currently migrating to new servers. The DNS will take a while to propagate, but there won't be any downtime during the move.
-
Tenboro
@ehentai
The server move was successful, and all five servers are back online.
-
Tenboro
@ehentai
Going to shut down the servers for the move. This means we'll be operating at somewhat lower capacity and redundancy for a few hours.
-
Tenboro
@ehentai
Tomorrow we'll be moving the first batch with five of our existing servers to the new rack. No actual site downtime is expected.
-
Tenboro
@ehentai
Fixed a problem with the wiki that prevented access unless you had an active login session.
-
Tenboro
@ehentai
Added some improvements and refinements to the Multi-Page Viewer. See patch thread for details.
-
Tenboro
@ehentai
Also, with the new cluster combined with improvements to the search engine, the gallery search limit is no longer necessary and was removed.
-
Tenboro
@ehentai
Incidentally, the improvements to the H@H distribution mechanism cut the load on the primary image servers by half. Success?
-
Tenboro
@ehentai
The archiver should now be more clever about picking servers for galleries that have an archived parent, to maximize image cache hit%.
-
Tenboro
@ehentai
Made some tweaks to the H@H distribution mechanism that should improve utilization for recently distributed files.
-
Tenboro
@ehentai
Fixed a bug with the uploader where a gallery could become inconsistent if an uploaded file was both corrupt and had a malformed filename.
-
Tenboro
@ehentai
Fixed some broken javascript stuff with the gallery uploader/manager.
-
Tenboro
@ehentai
New banners are up. Grats to aiwotorimodose, deep24froz, ShiroKitsune, ctxl, Tiap and Shuleria.
-
Tenboro
@ehentai
Happy new year to all the Asians and Europeans, and premature happy new year to all the Americans.
-
Tenboro
@ehentai
Updating one of the servers seems to have triggered a really weird CPU timing bug in the kernel, reverted to the old one for now.
-
Tenboro
@ehentai
Doing some maintenance on the old cluster nodes and database replicas. Shouldn't be causing any noticeable problems.
-
Tenboro
@ehentai
The switchover is now complete. Looking good so far, as far as I can tell.
-
Tenboro
@ehentai
About to start switching over to the new cluster. Note that you may be seeing various weirdness while it's transitioning.
-
Tenboro
@ehentai
All 8 nodes in the new primary web cluster are now replicating the live database. Barring any problems, the switch will be this weekend.
-
Tenboro
@ehentai
Fixed a lingering issue with the tracker that made it drop about 20-25% of all connections.
-
Tenboro
@ehentai
Seems like an overload caused by some search engine spider gone wild caused the tracker to go down. It should be back online now.
-
Tenboro
@ehentai
World still not ended in Europe. Also, the RAID rebuild finished at some point, and the server was put back live.
-
Tenboro
@ehentai
The RAID rebuild is only at 7.7%. It only takes about three hours under no load, so I'm failing it over to the backups so it can finish.
-
Tenboro
@ehentai
The new disk on the thumbnail server passed the test, and the RAID is (slowly) rebuilding.
-
Tenboro
@ehentai
Soo yeah, don't be an idiot, and don't download and run just anything you find on the internet.
-
Tenboro
@ehentai
I've received some reports lately that people have had their account details stolen, presumably after running some form of HV script/bot.
-
Tenboro
@ehentai
The disk was replaced, going to run an extended SMART test on it before rebuilding the RAID. It's running fine off 2/3 disks for now.
-
Tenboro
@ehentai
One of the new thumbnail servers is flagging some errors on a disk. Should be hot replaceable, but could be somewhat slower for a while.
-
Tenboro
@ehentai
These are all Xeon E3-1270V2s with 32GB RAM and a 120GB Intel 520 SSD. Now to do some burn-in testing.
-
Tenboro
@ehentai
Just finished the OS installation and core setup on the eight servers that will make up the new primary web cluster.
-
Tenboro
@ehentai
Switching in some new thumbnail servers. Which mostly consists of waiting for disk caches to warm up.
-
Tenboro
@ehentai
HV will be down briefly for a small update.
-
Tenboro
@ehentai
New servers now also live as lofi. And that ends this long-ass round of image/archive server upgrades.
-
Tenboro
@ehentai
The new servers are now live both as normal image servers and archivers. Started transferring the remaining lofi repo from the old servers.
-
Tenboro
@ehentai
The image repository finished transferring to the new servers already. Stuff happens a bit faster with gigabit. Bringing them online.
-
Tenboro
@ehentai
Curiously, even though the have twice the capacity and ten times the bandwidth, they ended up being cheaper than the servers they replace.
-
Tenboro
@ehentai
The second set of new image/archiver servers was delivered. These are identical to the previous set, with 12x3TB RAID-10 and Gbit uplink.
-
Tenboro
@ehentai
Pushed a tiny patch to Galleries that fixes a couple of minor issues with the standard image viewer and MPV.
-
Tenboro
@ehentai
Restarting a front-end node to apply some updates. You could be seeing some interruptions.
-
Tenboro
@ehentai
Large/cover thumbnails have been switched over to use a new set of servers. (And yes, we did this back in August too, but these are better.)
-
Tenboro
@ehentai
The new image server pair is now also online as lofi servers. Which means they are now fully deployed.
-
Tenboro
@ehentai
Two weeks later, the image/archiver combi servers finished syncing the image repo. Started bringing them online.
-
Tenboro
@ehentai
New banners are up. Grats to Death Grunty, ShiroKitsune, FLCLocust, ctxl, frostyb2003, aiwotorimodose and Dan31.
-
Tenboro
@ehentai
Pushed a smallish patch to Galleries that among other things re-adds the manual image size limitation.
-
Tenboro
@ehentai
Also applied a patch on the wiki for a potential but likely mostly harmless security vulnerability.
-
Tenboro
@ehentai
Starting the switchover to the new archivers.
-
Tenboro
@ehentai
They will also eventually act as image servers as well, but those are only about half way done transferring.
-
Tenboro
@ehentai
The new archive servers are almost ready to go live. They've been configured and tested, waiting for the final few hundred gigs of data.
-
Tenboro
@ehentai
Seems like the Firefox people realized just how hare-brained those changes were. As far as I understand, they will be reverted in 17.0.1.
-
Tenboro
@ehentai
"Uh, we shipped this?" Indeed, Kyle. Indeed.
-
Tenboro
@ehentai
Fixed retarded useragent change in Firefox 17 breaking certain forum functionality.
-
Tenboro
@ehentai
The patch was applied. You can find the patch notes at the usual place: forums.e-hentai.org/index.php?showtopic=98733
-
Tenboro
@ehentai
About to apply a new update to Galleries. Some minor turbulence might be expected.
-
Tenboro
@ehentai
They'll probably need the next week or so to transfer the image and archive repositories, after which they'll replace 4 old servers.
-
Tenboro
@ehentai
The new archiver/image servers were delivered a few hours ago. These consist of 2x gigabit servers with 12x 3TB disks in RAID-10.
-
Tenboro
@ehentai
Finally cracked a problem where Varnish would start dropping connections above 200 or so requests per second. Which has been a huge problem.
-
Tenboro
@ehentai
♪ Take these broken thumbs, and learn to serve again, learn to live so free~
-
Tenboro
@ehentai
Need to restart a front-end node. This could cause some load failures and slowness.
-
Tenboro
@ehentai
Put in some optimizations for the gallery search engine that should both reduce server load and increase the overall speed of searches.
-
Tenboro
@ehentai
Been playing with firewalls. One frontend node now runs a test setup that automatically blocks misbehaving hosts on the network level.
-
Tenboro
@ehentai
The patch should be fully live. Patch notes here: forums.e-hentai.org/index.php?showtopic=97758
-
Tenboro
@ehentai
Going to roll out a new version of Galleries. Standard live wing replacement of in-flight airplane disclaimer in effect.
-
Tenboro
@ehentai
The cluster should be fully stabilized again now.
-
Tenboro
@ehentai
Seeing a lot of network derping tonight, which is causing various issues.
-
Tenboro
@ehentai
You can be added to this group by a wiki admin (varst, Spectre, Maximum Joe or myself), or by any other creator.
-
Tenboro
@ehentai
In order to prevent spam, you can no longer create new pages on the Wiki unless you have been added to the new "Creators" group.
-
Tenboro
@ehentai
Fixed a reverse proxying issue that caused archive uploads to duplicate under certain conditions. Probably.
-
Tenboro
@ehentai
Had a short blip of downtime caused by some faulty routing. Seems to only have lasted a couple of minutes.
-
Tenboro
@ehentai
A new version of Galleries was released. You can read the patch notes here: forums.e-hentai.org/index.php?showtopic=96725
-
Tenboro
@ehentai
The faulty routing appears to have been resolved.
-
Tenboro
@ehentai
One of our satellite servers seem to be having some routing issues. This could cause missing thumbs/static content under some conditions.
-
Tenboro
@ehentai
Fixed a bug that made Galleries silently reject all-numeric tags even if they were pre-defined and in a namespace.
-
Tenboro
@ehentai
This caused caches to cool, which made it worse. After some rebalancing, the site was able to recover. At least it didn't desync this time.
-
Tenboro
@ehentai
Seems to be fully stabilized. Basically the cleared cache caused overload on one of the database replicas, which eventually made stuff fail.
-
Tenboro
@ehentai
Taking the site offline for a second to help it clear out.
-
Tenboro
@ehentai
One of the databases didn't like the additional load very much. It should stabilize in a little while.
-
Tenboro
@ehentai
Restarting another cluster node. The desyncing issue should be fixed, but we'll see what happens this time.
-
Tenboro
@ehentai
Some desyncing issues did cause a few H@H clients to be knocked offline. I'll have to address that before doing the rest.
-
Tenboro
@ehentai
Going to restart one of the primary web cluster nodes. You could see some minor interruptions as stuff is rerouted.
-
Tenboro
@ehentai
Considering that corruptions on this RAID caused the site to be in read-only for more than a week six months back, that's some lucky timing.
-
Tenboro
@ehentai
Was clearing out some stuff on the old uploader, and noticed that the RAID started throwing some errors on Oct 22 - 2 days after the switch.
-
Tenboro
@ehentai
Oh hai there, new all-time peak traffic.
-
Tenboro
@ehentai
Linking thumbnails to other galleries in gallery comments should now be working again.
-
Tenboro
@ehentai
The system finished generating combine thumbnails for pages 41-100. Moving on to 101-200, we'll probably stop after that.
-
Tenboro
@ehentai
The H@H file distribution is just about fully balanced out again after that distribution bug, at least load-wise.
-
Tenboro
@ehentai
New banners are up. Grats to Kinsei, Handsome Phil, Death Grunty, ShiroKitsune, frostyb2003, Dan31 and FLCLocust.
-
Tenboro
@ehentai
Normal and combine thumbs are now fully online on the new thumbnail servers. The load is looking great so far.
-
Tenboro
@ehentai
About to start bringing the new thumbnail servers online.
-
Tenboro
@ehentai
Pushed a tiny patch to Galleries with some minor refinements to the Multi-Page Viewer.
-
Tenboro
@ehentai
Spoke too soon. The servers now have all the working IPs they need, so now they just need that final setup and a thumb transfer.
-
Tenboro
@ehentai
The disk issues have been resolved. The IP issues, not so much. Currently loading all the thumbs to the servers while waiting for that.
-
Tenboro
@ehentai
Early testing uncovered both some issues with a few of the disks as well as some IP routing issues. Currently waiting to get those resolved.
-
Tenboro
@ehentai
The new servers were delivered. Going to start initial testing and setup.
-
Tenboro
@ehentai
The system should rebalance fairly quickly, but there will still be an underutilization of H@H until it does.
-
Tenboro
@ehentai
Basically, a string being silently cast to a double by the database caused the mechanism to only consider data from one single image server.
-
Tenboro
@ehentai
Fixed a bug in the H@H file distribution mechanism introduced mid-October that caused severe imbalance of which files were distributed.
-
Tenboro
@ehentai
After more tweaking, thumbs are now temporarily distributed 50% between the old set and the repurposed one. New servers tomorrow, hopefully.
-
Tenboro
@ehentai
Two days of heavy optimizing and load rebalancing seems to have done the trick as far as 503s go. We'll know in a few hours.
-
Tenboro
@ehentai
Ordered four new servers that should be able to cope better with that specific thumbnail load. And that are also gigabit.
-
Tenboro
@ehentai
For now, I've tasked the system with further extending the range of combine thumbs to include pages 41-100, but that will take some time.
-
Tenboro
@ehentai
Even after fully warming up caches, the repurposed thumbnail servers were unable to sustain the load. I'll have to work out something else.
-
Tenboro
@ehentai
The last of the thumbnail servers finished restructuring their storage overnight. Bringing them back online now.
-
Tenboro
@ehentai
Going to restart some of the frontends. While you should automatically fail over to another one, it could cause some short interruptions.
-
Tenboro
@ehentai
Currently working on switching around some of the thumbnail hosting. This could cause somewhat slower thumbnails for the duration.
-
Tenboro
@ehentai
Hopefully this should resolve some of the poor routing we've been experiencing over the last couple of days.
-
Tenboro
@ehentai
According to our hosting provider, additional network connectivity will be added within the next 24 hours.
-
Tenboro
@ehentai
Tweaked the calculations for alignment of combine thumbnails to try to prevent some cases of them bleeding into each other.
-
Tenboro
@ehentai
Uploading has now been enabled on the new uploaders. I'll be monitoring them over the next few hours to make sure they work as intended.
-
Tenboro
@ehentai
The sync has finished, just need to do some final relinking and testing.
-
Tenboro
@ehentai
Uploads are now turned off. Starting the migration process.
-
Tenboro
@ehentai
I'm planning on performing the upload server migration around 0830 UTC (in 2 hours). Gallery uploading will be turned off for a few hours.
-
Tenboro
@ehentai
Some internal network routing issues seem to have caused a bit of an outage about half an hour ago. The site recovered by itself.
-
Tenboro
@ehentai
Fixed the twitter feed not updating on the front/news page. Yay Twitter for randomly changing the API url.
-
Tenboro
@ehentai
The redundant upload mechanism still needs some testing, but it should be ready in time. The actual process will probably take a few hours.
-
Tenboro
@ehentai
3 days later, the new servers are holding together well. That done, I'm planning on moving on with the upload server migration this weekend.
-
Tenboro
@ehentai
The new server is now online as a lofi server. That means the 3 old lofi servers are now phased out in favor of the 2 higher-capacity ones.
-
Tenboro
@ehentai
The server is now fully live. Looking good so far.
-
Tenboro
@ehentai
It should be going live for normal images within the next fifteen minutes. lo-fi images should follow tomorrow.
-
Tenboro
@ehentai
The second of the new image servers from nine days ago recently finished the initial sync and is almost done with the catchup sync.
-
Tenboro
@ehentai
Galleries 0.21 was released. This adds the new Multi-Page Viewer.
-
Tenboro
@ehentai
The query load per second went from about 1200 to 300. Which should help quite a bit.
-
Tenboro
@ehentai
Put in a small update to reduce the load on the stat/state tracking server and improve resilience if it goes down.
-
Tenboro
@ehentai
Should be returning to normal now. Note that certain volatile data (like HV battles) would have been lost.
-
Tenboro
@ehentai
A core server seems to have entered some kind of weird semi-locked up state. Currently trying to recover it.
-
Tenboro
@ehentai
Finished all the setup on the new server, and it's well under way transferring its half of the repository.
-
Tenboro
@ehentai
The second new image/lofi server was just delivered. Like its twin, this is a Dual E5620 with 12x2TB RAID-10 and 100TB @ 1 Gbps.
-
Tenboro
@ehentai
Posted a small update to Galleries that allows use of both combine and normal thumbs on the same page, for when more than 4 rows are shown.
-
Tenboro
@ehentai
New banners are up. Grats to The Argent Trigger, Handsome Phil, FLCLocus, aiwotorimodose, Zumato, Kinsei and Tenseigamoon.
-
Tenboro
@ehentai
The cleanup dropped the number of unique tags from 238998 to 87104. Those 150k and change were unused and made up only 3% of total tagging.
-
Tenboro
@ehentai
Running a script to clean up a bunch of unused tag identifiers. Shouldn't really take very long or be noticeable.
-
Tenboro
@ehentai
Seems to be cleared up.
-
Tenboro
@ehentai
One of our hosts had a router failure which affected about five of our servers. One is still unavailable, but this shouldn't be noticeable.
-
Tenboro
@ehentai
This finished a few days ago. I now pushed an update to actually use them. Which should make it both faster and use less resources.
-
Tenboro
@ehentai
Waay back, around Aug 25th, I started a process to, among other things, build combine thumbnails for thumb 21-40 of all galleries.
-
Tenboro
@ehentai
HV will be down briefly for a smallish update.
-
Tenboro
@ehentai
Ordered the second new image/lofi server. Will probably be delivered mid next week, unless they happen to have one ready to go.
-
Tenboro
@ehentai
Fixed another bug with the recent patch where it would fail to preserve the filename of images when attempting to save them.
-
Tenboro
@ehentai
It's not technically necessary to move the lofi repositories around since they are just regenerated, but it avoids overloading new servers.
-
Tenboro
@ehentai
The new image server's share of the 980x lofis have finished moving over, and are now being served from the new server.
-
Tenboro
@ehentai
Fixed some issues in the new release that could crop up under certain conditions. (Like running dry your image allotment.)
-
Tenboro
@ehentai
(By both of them I mean the new image servers. Each will do half of the lofi images. 980x ones were previously offloaded to another server.)
-
Tenboro
@ehentai
Also waiting for the new server's share of the 980x lofis to sync over. When both of them are fully live, I can reenable the spare database.
-
Tenboro
@ehentai
Put in some optimizations for which image servers are used when, in order to better balance the load for those with other tasks as well.
-
Tenboro
@ehentai
The new server is now also live as lofi. It's performing great so far.
-
Tenboro
@ehentai
And it's live, for normal images anyway. I'll be evaluating its performance over the next few days before ordering the second one.
-
Tenboro
@ehentai
The new image/planned-as-archiver-but-for-various-reasons-will-be-lofi-instead server is just about done syncing, probably goes live today.
-
Tenboro
@ehentai
The server is back online. Which means 980x lofi is available again.
-
Tenboro
@ehentai
The file move is at 50%. Looking at three hours and change.
-
Tenboro
@ehentai
Finished all the setup and stuff, still waiting for the file transfer to finish. It's at ~25%, so it'll probably take ~5 more hours.
-
Tenboro
@ehentai
The OS has been reinstalled. Currently moving the files back to their intended location and doing some final setup.
-
Tenboro
@ehentai
Taking the image/lofi server with the borked RPM database offline to reinstall it. 980x lofis will be unavailable for a few hours.
-
Tenboro
@ehentai
Pushed a smallish update to Galleries that among other things add a one-liner from any uploader comments back to the gallery info pane.
-
Tenboro
@ehentai
... there are only two kinds of people: those who don't know they are using Internet Explorer, and those who know what a web browser is.
-
Tenboro
@ehentai
Major 0-day exploit in the wild for Internet Explorer, techies and governments everywhere recommend against using it. Only problem is...
-
Tenboro
@ehentai
Started syncing the repository to the new server. These will hold the third master set when the old uploader is finally phased out.
-
Tenboro
@ehentai
The first of two new image/archiver servers was just delivered. These have 12x 2TB drives in RAID-10, and 100TB@1Gbit bandwidth. For fun.
-
Tenboro
@ehentai
The patch was applied without any major issues. Full patch notes here: forums.e-hentai.org/index.php?showtopic=88775
-
Tenboro
@ehentai
About to apply a major update to Galleries. You may be seeing some weirdness like inability to tag/rate galleries while it's being applied.
-
Tenboro
@ehentai
I need to make a temporary clone of a file repository before actually wiping it, and that's estimated at 16 hours. Tomorrow then.
-
Tenboro
@ehentai
One server managed to hose its RPM database due to a failed update, going to wipe and reinstall that one in a bit. 980x lofis will be down.
-
Tenboro
@ehentai
Fun times ahead. Might be time to get that IPv6 support up and running.
-
Tenboro
@ehentai
In other news, RIPE is at stage 3 IPv4 exhaustion, so no one can get more than 1024 more IPs and no new provider can be assigned IPv4 space.
-
Tenboro
@ehentai
Aand put in some even more aggressive pruning. These should also make the search results more accurate.
-
Tenboro
@ehentai
Put in a small update that will make the search engine better at pruning tags that can never get any hits anyway, which speeds up searches.
-
Tenboro
@ehentai
Network's being kinda shit tonight. But it's intermittent and kinda hard to pinpoint exactly.
-
Tenboro
@ehentai
A network outage caused the site to be partially or fully unavailable for ~11 minutes roughly six hours ago. Seems to be all clear, though.
-
Tenboro
@ehentai
So if you still can't connect, try switching to Google's DNS, which is 8.8.8.8 and 8.8.4.4
-
Tenboro
@ehentai
Tweaking the TTL on the domains seems to have helped, but you could still be seeing issues, depending on what DNS servers you are using.
-
Tenboro
@ehentai
The domain provider and DNS host that has most of our sites is currently under a massive DDoS, which could make the site appear offline.
-
Tenboro
@ehentai
One of our hosts is having some connectivity issues, which could result in dropped connections and lower than normal performance.
-
Tenboro
@ehentai
Fixed a couple of bugs related to the new tag namespacing feature.
-
Tenboro
@ehentai
Fixed a problem that prevented people from adding images to bounties.
-
Tenboro
@ehentai
Going to restart some of the thumbnail servers. You could see short periods of it being slower than normal as the load is rerouted.
-
Tenboro
@ehentai
Fixed a configuration issue that left the Atom feeds uncompressed even if Accept-Encoding:gzip was used.
-
Tenboro
@ehentai
Tag toplist processing has been reenabled. It is now delayed by three days, and only gives points for valid and non-redundant tag votes.
-
Tenboro
@ehentai
New banners are up. Grats to Golden Sun, aiwotorimodose, Tiap, Panuru and Dan31.
-
Tenboro
@ehentai
DNS is currently propagating to switch to different C&Cs, and all clients should have been triggered to update with the new IPs.
-
Tenboro
@ehentai
Seeing a lot of packetloss that seems to be localized to the public link on the H@H C&C server. Working on moving it to another server.
-
Tenboro
@ehentai
Seeing some minor packetloss unrelated to the missing H@H clients, but at least the image servers are coping with the load by now.
-
Tenboro
@ehentai
The image servers are still running at close to capacity, at eight times the normal load, so if images seem slow to load, this is why.
-
Tenboro
@ehentai
Either that or it just hangs completely. It's recovering again, but this caused a lot of H@H clients to crap out, might wanna check yours.
-
Tenboro
@ehentai
The database decided it would like some me time for the final table rename, should finish in not too long.
-
Tenboro
@ehentai
Tag table rebuild is at about 40%. Roughly on schedule.
-
Tenboro
@ehentai
The table rebuild will take approximately three hours.
-
Tenboro
@ehentai
Tagging will be temporarily unavailable while I restructure some core database tables.
-
Tenboro
@ehentai
Got the monthly uptime report from our monitoring service. Galleries had 100% uptime in August. Knock on wood, touch iron, etc.
-
Tenboro
@ehentai
The change has been fully applied. Caches are still warming up, but performance seem to have increased by a fair bit.
-
Tenboro
@ehentai
About to apply a change that changes how the system deals with "cover" thumbnails. This is a bit experimental.
-
Tenboro
@ehentai
Switching around some of the static content servers. Should be pretty painless.
-
Tenboro
@ehentai
Finished the initial sync on the four new gigabit thumbnail servers, started the catchup sync. I suspect they'll enter service this weekend.
-
Tenboro
@ehentai
Applied a quick security patch to the wiki. Nothing super serious, just a couple of potential CSRF and XSS vulnerabilities.
-
Tenboro
@ehentai
And it's back online.
-
Tenboro
@ehentai
The battlestate and logging server will be offline briefly while I apply some configuration changes.
-
Tenboro
@ehentai
The tracker frontend crapped out for some reason or another, should be back online now.
-
Tenboro
@ehentai
I like it when you order a few servers before hitting the shower, and they are deployed and ready before you've finished drying.