1. The server in Open Beta Test Status
    All data will be WIPED when beta stage is finished

Solved We failed. But can fix mostly everything with your help.

Discussion in 'News and Updates' started by Arrow, May 6, 2018.

Thread Status:
Not open for further replies.
  1. Arrow

    Arrow Administrator Staff Member

    GM
    Let me explain what happened.
    We decided to stop this absurd and shameful situation around launching MegalithRO today.
    We must declare - we have failed today. This product is not acceptable by any means.

    With your help, we will present quality product from start, without childish bugs and issues which made from sleepy head, without laggy forum and site, without downtimes in the day of the opening by 3rd party provider, and so on.

    We tried in hurry to pack the server without correct deep testing.
    This is absolutely not acceptable with features what we are going to offer.

    Several times postponed the server opening, and played with your wishes and patience.
    This is vulgar, very rude and harming the attention, ideas and patience given to us from players.
    Shame on us.

    We can't be impatient and don't care about bugs, about problems, about mistakes, about managing the community, about the technical side, empty promises and everything else.

    The whole day absolutely everything was against us. No, seriously.
    Every component did not work, which earlier did well. Everything said to us -> do not open the server today.

    1. Customized prontera map, for years without issues, and today it starts to have annoying glitches, which forced us to replace map 1 day before server opening.
    2. Then we faced tons of problems with git environment, broken repositories, broken chunks, continuously and nonstop issues with conflicts which never happened beforer. Every new commit and change was a very big pain. Thx to git environment, we've missed several features and stucked a lot for resolving issues which we do not foreseen.
    3. Then a pre-configured and tuned website starts to lag as I don't know what, we did stress testing under heavy loads, and everything was perfectly fine, but today, it was not.
    4. Then we realize that the opening countdown timer is incorrect, and broken, and display to us totally different time what we expect, which force us to do much more mistakes.
    5. Then, when all problems solved, 5 minutes before the server opening go offline whole DC where we host our webhosting. The hosting provider for at least 3 years NEVER got any performance downgrades or connection issues, but today it was not on our side.
    6. They solved the issue in around 1 hour, and this 1 hour we need for taking a database from webhosting and replace and merge it with the database on the game-server. So extra postponing the server opening and whispering words "do not open it today" happen again.
    7. DC back online, and we have 30 minutes before server opening, and we need to take a data from webhosting, transfer it to game one, and merge data structures which are different together for making it work. And what do you think? Stable internet connection goes to 12-15kb/s. Is not it funny?
    8. Okay, somehow we delivered website db to game host and have fully working game db where are no issues so far for more than few months of development, and expected that we will easy like always will merge it. And you know what? Around 20 times mysql app stucked, froze, and did not work just for executing one simple command.
    9. Somehow the problem with the db solved, we satisfied how db merged, and looks like the hardest time behind. But it's not. At this point, we have NO connection between our game database and website. Users not touched, firewall works fine, security tools did not give any effect, with tcpdump we see that packets was not routed at all.
    10. After around 30-40 minutes with trying different methods to avoid the situation (imagine, you have whole developments, backups, monitoring, delivery services, proxy system environment and so on, and you just don't have internet connection from one host to another for launching the server).
    11. After around 40-50 minutes of failed attempts to recover the situation, we decided to reinstall the whole environment, and you know, this helps. What I tell you here is not only about a simple distro re-installation, I talk about advanced routing, backups, monitoring, security tools, a practice which require a lot of time.
    12. Okay, the connection between web hosting and game-server stabilized, but with tons of lags, and problems. Do you think this is DDoS attacks? Not really, they are absorbed. This is just issues in both DC at the same time with a network of administrative routers.
    13. Okay finally we made to work the server, connection repaired, the whole environment re-installed, few hours wasted, and the game server opening. At console before the server opening silence, no errors, no warnings, no any problems at all. And when new players join the game server we watch the massive wave of errors which never happened before for years, we only today noticed about these errors. And boom crash. Okay, no problem (of course it is!), let it debug on the live envelopment, let's run a debugger and check what causing the error. Okay found, except the crashes, we see tons of errors, which again, never happened before. Ok, sh*t happens, we can collect all of these and fix them all together. One of the biggest memory leaks in my life caused sprite errors and way to equip a knife on an Acolyte. We started to search what sprite is it until found that the sprite never exists, and never should be. This is a side effect of memory leak and wave of errors. Somehow we figured out the things that potentially can do the trick, and added (I'm ashamed on myself) some fixes to avoid any similar issues in the future.
    14. Collected and fixed many small different bugs that did not fixed earlier for some reason because of a hurry.
    15. Starting the server. And after 15-30 minutes I'm reading reports and PM about different bugs. At the same time, my phone pressing a key on the keyboard which is connected to console where is the server working. This is resulting in a lag on the server, because of such actions (keyboard manipulate with console output and freeze it).
    16. We expected that the wave of errors will go, and crashes will be fixed, and to be clear it is true, crashes fixed, but another problems still exist, and do massive performance degradation and overall feeling about what is going on.
    17. Then someone starts to report about many worst bugs what personally I never faced ever maybe for at least 7-8 years, these bugs totally destroy the whole server in the very short perspective (few days). Because we technically can fix it, but we don't know when, and the results of these mistakes (this is totally my fault) = 99\10 levels, different classes, different items and so on. This makes a mess at the solving the situation on the server which is still 2 hours old.
    18. So, as you understand, we did not sleep for around 48 hours for sure. Did a lot of mistakes, did not test the server throughly, and did not know about very strange and powerful bugs what we already know for now.

    For what is this wall of text?
    The main purpose is to explain that even under heavy stress without sleep and rest and against everything, we did not give up and tried to make the server work today, but situation goes worse and worse. As the result, the DB become a big mess and will be wiped out. The server is buggy and will be fixed. The whole project is raw and must be made more professional and responsible. Just try to understand, that we tried our best to keep the server online as maximum, but the situation and our own responsibility will not allow us to keep continue this absurd condition. So we decided to temporarily close the server for some short period of time.

    What's next?
    Out the box, we plan to present quality product based on players wishes, suggestions, and propositions, bug-free, and without such big mistakes what you saw today.

    What does it mean?
    That means, we must change our plans and be much closer to the community and ask for help for making the server of our and your dream in a very short period of time. Because without you, we are nothing.

    What is the plan?
    1. Analyze collected information and fix wave of errors
    2. Open CBT for people interested in the project and check the features for finding and fixing bugs before release day.
    3. With testers, we will be able to collect whole information much faster and fix it.
    4. With responsible testers, we can make a server based on objective good things what is required by the community.
    5. Open well tested, bugs free, without a headache and based on the community for community server with tons of features what we offer.

    How start to be beta-tester?
    1. You already have a game client
    2. You already have an account
    3. We already have the server
    4. We will continue fixing issues and implement your suggestion, you will continue playing.
    5. At date X when the found issues will be fixed, and requested features implemented, we will announce the server opening. But before that, we will prepare and will make sure that our 3rd party provider, phone, keyboard, lack of sleep, the internet connection will not disturb.
    6. Then, before opening, we will do a wipe, and release the server that you will enjoy so we all can be satisfied.


    [​IMG]
     
    akaineko likes this.
  2. JayLinh

    JayLinh Member

    nobody wants to help you.

    1. we asked you to used discord, you declined.
    2. you refused to communicate with players.
    3. after a 2 month wait period, your product was not suitable for release.
    4. as players made suggestions, you shut down the server.
    5. ????
    6. Profit
     
    O BRUXO, Jowy4u, MeoCon and 1 other person like this.
  3. Eviy

    Eviy New Member

    AzureSky likes this.
  4. thegoodgood3222

    thegoodgood3222 New Member

    RIP
     
  5. Kiryuu

    Kiryuu New Member

    So, if I understand correctly.

    1. Leveling that people have done = total waste of time, because DB is a mess and needs to be reseted
    2. Beta testers who will be kind to stay, and to help you, will also get a wipe in their face after the CBT?
     
  6. Fluffy

    Fluffy Member

    Then, before opening, we will do a wipe, and release the server that you will enjoy so we all can be satisfied.[​IMG]

    Should have been done this previous month. Too little too late, thanks for the update bye.
     
  7. Andy'

    Andy' New Member

    o_O:mad::(
     
  8. PAKOT

    PAKOT New Member

    Of course, I understand everything, but as a person who was engaged in Ragnarok shards at the beginning of zero, I can not understand how it is possible to transfer the start of the server in general for more than 2 months, and not to eliminate children's errors, where the channel reservation? Where is the server cluster with server migration between cluster machines in real time? Where is the database backup? How could you test the server under load and not detect errors? Admin do not be offended but the feeling is created that you first downloaded the emulator and try to run it.
     
    Teban and JayLinh like this.
  9. xstormrage

    xstormrage Poring Licker

  10. Persib Mania

    Persib Mania loyalitas tanpa batas

    lucky player get free job change during jobchanger error :rolleyes::cool::cool:
     
  11. Kurapika

    Kurapika New Member

     
  12. xstormrage

    xstormrage Poring Licker

    even if you do quest, you should still be able see the exp gain.
     
  13. Kenny

    Kenny New Member

    Keep your heads up. It can be a lot of work. Don't get discouraged. If you're really trying to fix the problems and create a great environment for a community to grow I'm in. Otherwise it would be better to let that ship sink and start again when some time has passed.
     
    Sunnivah and Apple Juice like this.
  14. Jellybean

    Jellybean New Member

    all this server did was giving us false hope..
     
  15. Infidelities

    Infidelities New Member

    got promo message about ro new ps
    decided to give it a try
    downloaded the client + updated it
    day-1 make char, cant login to game
    day-2 trying to play, still novice, trying to find suitable hunting ground
    get into morroc field 10, walking around *where are the enemies?*
    decided to change map *no portal*
    decided to go back to town *didnt bring butterfly wing*
    decided to go through morroc gate *no portal either*

    cant hunt, cant go back to town, cant suicide, cant do anything

    so...
    thanks and bye
     
  16. Persib Mania

    Persib Mania loyalitas tanpa batas

    @go 0?
     
  17. Thrace

    Thrace New Member

    I'll help beta test. I don't mind.
    Some people like to whine. The only thing I ever found whining good for was to get my bottle or pacifier. I haven't used either in decades so why whine now.
     
  18. nescafe

    nescafe New Member

    fucking meme server
    i dont think anyone with brain will believe in this server again.
     
    Last edited: May 6, 2018
  19. Barbie

    Barbie New Member

    Had like 2 month to prepare for opening , got delayed, why should we trust that you will fix everything ?
    How much time it will take 3, months more ?
    NO TY
     
  20. Nutty

    Nutty New Member

    Ye, we done here
     
Thread Status:
Not open for further replies.