z

Young Writers Society


How YWS, WFP, and the Chatroom Work



User avatar
425 Reviews



Gender: Male
Points: 11417
Reviews: 425
Mon Aug 17, 2015 2:21 pm
View Likes
Nate says...



How the Young Writers Society, WriterFeedPad, and the Chatroom Work


Everything you never wanted to know

I'm not quite sure if really anyone is interested in what makes YWS tick, but I do see from time to time questions about the underlying infrastructure, even if they're not phrased that way. For example, there's the "Why is WFP up if YWS is down?" question, or the "Why can't I log into chat using my YWS account details when I get the login box?"

So in an attempt to answer these questions and more, here's an overview of the underling infrastructure for each of the major components of YWS!


The Young Writers Society -- Server

YWS runs on a LAMP server, meaning that it uses Linux, Apache, MySQL, and PHP. This is a popular kind of setup, and most sites you visit probably use something similar. Linux is the operating system, and YWS runs on Ubuntu 14.04. Apache is the web server process. MySQL is the database, and PHP is the scripting language used for web development.

The server is hosted on an Digital Ocean VPS with 8GB of RAM. A VPS is a virtual private server, which for all intents and purposes is really no different from a dedicated computer. Along with YWS, I also run several other websites off of this server, including Writing Gooder (but not WriterFeedPad as explained below).

Whenever you visit YWS, your request first goes through Cloudflare. Cloudflare is a popular reverse proxy service that helps to block malicious content, such as malware, from reaching YWS. It also acts as a content delivery network, which means it caches YWS' static files (images, javascript, stylesheets) around the world so that the site is as fast as possible. For example, say you live in Sydney. Normally when you visit YWS, you'd have to download all the site content from the server in New York. That typically adds a few to several seconds to load times. Cloudflare reduces that by instead hosting that static content on a server in between (possibly even in Sydney itself).

Sometimes that process breaks down either because there's a problem with Cloudflare or a problem with YWS. When this happens, you get the Cloudflare error message.


The Young Writers Society -- Site

YWS itself runs on top of phpBB 3.0.14. PhpBB is a type of forum software, and YWS' usage of it is most clear on the forums page. However, YWS relies on it for pretty much everything.

A few may know that phpBB recently released 3.1, and plans to release 3.2 this Fall. YWS will never be upgrading to those, and will be sticking with the 3.0 series.


WriterFeedPad

WFP is on a completely separate server from YWS, which is why one can stay up even when the other goes down. The syncing of user accounts between the two is due to some trickery on my end. Once an hour, a server-side script on YWS checks for any changes to user accounts. If there has been a change, then the script sends a notice of that change to WFP's database.

Other than this occasional syncing, there is no link between the WFP server and the YWS server. The WFP server itself is a LAMJ server: Linux, Apache, MySQL, and Java. Like YWS, it runs on Ubuntu 14.04.


WriterFeedPad -- Site

WFP uses an older version of Etherpad, which is an open-source document editor and the original basis for Google Docs. There is a newer version of Etherpad available called Etherpad-Lite, but the difference between the two is like night and day. The biggest difference, though, is the lack of user accounts with Etherpad-Lite, which is pretty much a dealbreaker. Technically, there is a plugin that adds user accounts, but it's designed with tight working groups in mind rather than with separate individuals.

For the most part, WFP's Etherpad has not been modified in any way. The exception is with the login system. PhpBB passwords are encrypted using the phppass framework, so in order to allow users to use the same password on YWS and WFP, I had to replace the Etherpad password authentication system with one that's built around phppass.


YWS Chatroom

The YWS chatroom runs on the YWS server, but for all intents and purposes, it's a completely separate entity from YWS. The chat program used is called CANDY (chats are not dead yet) chat, and it's an XMPP client (XMPP is an instant messaging protocol kind of like how HTTP is a world wide web protocol).

Since it uses XMPP, you don't technically need to be on YWS in order to use the chat. You can also connect to it using an IM client such as Pidgin. However, I do hide how to do this as the web chat and the IM client chat aren't fully compatible. In particular, the moderation tools are completely different (and will differ even from IM client to IM client).

In any case, the chat and YWS itself are two completely different things -- similar to how YWS and WFP are two different things. Besides YWS running on HTTP and chat running on XMPP, the underlying infrastructure is different too. YWS uses PHP and MySQL while the chat uses erlang and mnesia.

In order to make the two talk so that you can be automatically logged in under your YWS account, a separate account is created for you on chat. But as phpBB passwords are encrypted and there exists no phppass framework for erlang, a key is automatically generated for your chat account. So what ends up happening is that phpBB handles the user authentication part and then submits a key to the chat that allows you to automatically login.

That's why when you get the login box in chat, your YWS credentials don't work. It's a completely different entity.


Questions?

If you have any questions, just let me know!
  





User avatar
170 Reviews



Gender: Male
Points: 0
Reviews: 170
Mon Aug 17, 2015 6:16 pm
deleted5 says...



Interesting topic! One question:
the server is hosted on an Digital Ocean VPS with 8GB of RAM.

I always thought 8gb of ram was quite little for a server. My pc itself has 8gb. Does YWS get anywhere near needing to use all 8gbs?
I AM YOUR GOD. -AlexSushiDog
Checkmate Atheists.
  





User avatar
425 Reviews



Gender: Male
Points: 11417
Reviews: 425
Mon Aug 17, 2015 7:09 pm
Nate says...



@AlexSushiDog ->> I like your question!

For YWS, 8GB is enough. Generally speaking, the server is using 6.5GB of RAM with 4.5GB of that devoted to the database and most of the rest to Apache. The only times it pushes up against the limits is during the nightly backups that happen at 4am New York time.

In fact, YWS could probably make due with just 4GB of RAM, although I'd have to trim the database to 1GB in size to make that work (could get there by deleting all PMs, works, and posts from prior to 2012, but that'd be an unpopular decision... to say the least!).

Concerning web servers in general, most websites of YWS' size and smaller don't need a lot of RAM. This is because they don't need to run graphics-heavy applications. Instead, what's more important is processing power, a good hard drive, and a solid network.

On another note, the WriterFeedPad server only has 1GB of RAM, which is not enough. About once every couple months, it'll crash because it's run out of memory.
  





User avatar
170 Reviews



Gender: Male
Points: 0
Reviews: 170
Mon Aug 17, 2015 7:18 pm
deleted5 says...



So in the future you might need to increase RAM size or delete some stuff?
I AM YOUR GOD. -AlexSushiDog
Checkmate Atheists.
  





User avatar
425 Reviews



Gender: Male
Points: 11417
Reviews: 425
Mon Aug 17, 2015 7:53 pm
Nate says...



Aye, eventually YWS will indeed need to either upgrade to a 12GB server (the next size up) or stuff will need to be deleted from the database. The first option is the latter as there is already at least 500mb that can be removed safely. But that just buys time, and it's already getting close to the point where I'll need to do that.

For the moment, I think the current server is probably enough for at least another two years, although that could change. When I do upgrade the server though, I'll probably split it into two. One would be just for the database (8GB) while the other would be for everything else (4GB).
  





User avatar
472 Reviews



Gender: Male
Points: 25
Reviews: 472
Tue Aug 18, 2015 3:28 pm
Lightsong says...



Thanks for the info! :)
"Writing, though, belongs first to the writer, and then to the reader, to the world.

The subject is a catalyst, a character, but our responsibility is, has to be, to the work."

- David L. Ulin
  








"I wish we could all get along like we used to in middle school... I wish I could bake a cake filled with rainbows and smiles and everyone would eat and be happy..."
— Unnamed Girl from "Mean Girls"