Author Topic: A note on Sunday's server downtime  (Read 1080 times)

Offline josephd

  • Administrator
  • Member
  • *****
  • Posts: 763
  • Karma: 270
  • Peppermint OS Website Admin
    • View Profile
    • LinuxBookPro
  • Peppermint version(s): Peppermint 10
A note on Sunday's server downtime
« on: March 01, 2020, 11:11:03 pm »
Hi everyone,

During multiple attempts to renew the security certificate the Peppermint web server crashed. This resulted in the need to restore our from a backups made around 4 UTC yesterday. Any posts, DMs in have been lost as a result.

We're currently running the web server on Ubuntu 14.04 which as you know reached end of life last year. Three separate attempts to perform in place upgrades to 16.04 resulted in MySQL errors. Although ironically updated the encryption certificate for about 15 minutes until the server crashed near the end of the process.

I will need to research this issue a little further before making another attempt. In the meantime please be aware that traffic on this forum and the Peppermint OS website may not be encrypted or has weak encryption. That basically means your surfing of this website could be intercepted by third parties. Particularly if you're on a open wifi network.

I apologize for the lost posts and DMs made yesterday.

Thank you for your understanding as I attempt to resolve these issues.
« Last Edit: March 01, 2020, 11:15:12 pm by josephd »
Joseph Dickson | www.linuxbookpro.com | www.joseph-dickson.com | https://twitter.com/joe4ska - "I also enjoy Debian Stable, Pop!_OS, and Trisquel."

Offline stevesveryown

  • Trusted User
  • Member
  • *****
  • Posts: 423
  • Karma: 173
  • Peppermint Fan
    • View Profile
    • stevesveryown
  • Peppermint version(s): Peppermint 10
Re: A note on Sunday's server downtime
« Reply #1 on: March 02, 2020, 06:01:38 am »
Joseph,

Thanks for the update.  It is unfortunate that things like this keep happening.  On my YouTube Channel I am being asked if this is the end for Peppermint.  I keep telling them to hold on and give the team a chance to recover from their loss.  Also some big proponents of Peppermint in the YouTube community have already jumped ship.  I will be here and patiently await to see what happens.  If there is anything you would like passed on to the YouTube community please let me know.

Offline The Omen

  • Member
  • ***
  • Posts: 143
  • Karma: 31
  • New Forum User
    • View Profile
  • Peppermint version(s): Peppermint 9 respin
Re: A note on Sunday's server downtime
« Reply #2 on: March 02, 2020, 09:40:51 am »
Hi folks,

I, for one, am going to stick with Peppermint as I realize that anytime there is a "death in the family", that things are never going to be quite the same ever again and there are always adjustment problems to deal with.

This is just one of those adjustment problems is all.

Mr. Greaves would want us all to stick it out and be supportive.

This is not doomsday or the end of Peppermint, just an adjustment for all of us to "keep calm and carry on" while everyone is adjusting to the new situations as they arise and hopefully learning that the "keys to the kingdom" need to be held not by just one person but also entrusted to others so that smoother transitions can take place in future.

Warm Regards to all, especially those that are now taking the lead in rectifying these otherwise minor teething problems as we all need to grow and accept the situation as it is.

The Omen




Offline spence

  • Administrator
  • Hero
  • *****
  • Posts: 3031
  • Karma: 345
  • peppermint user since 2010
    • View Profile
  • Peppermint version(s): Peppermint 10 Respin
Re: A note on Sunday's server downtime
« Reply #3 on: March 02, 2020, 10:50:33 am »
Also some big proponents of Peppermint in the YouTube community have already jumped ship.  I will be here and patiently await to see what happens.

Might you alert us to who is jumping ship so the rest of #teampeppermint may make contact @stevesveryown? 

 ;)
spence
PeppermintOS 10 Respin currently installed  on:
'16 Antec Aria rebuild
& '18 Asus VivoBook


Do not despair, grasshopper...
    with patience all will be revealed...
       Through pain, enlightenment will come.

Offline josephd

  • Administrator
  • Member
  • *****
  • Posts: 763
  • Karma: 270
  • Peppermint OS Website Admin
    • View Profile
    • LinuxBookPro
  • Peppermint version(s): Peppermint 10
Re: A note on Sunday's server downtime
« Reply #4 on: March 02, 2020, 11:12:36 am »
Thanks for your continued support @stevesveryown. Mark was very good at what he did in this project and no one person can replace such a dedicated project lead. We didn't have a second person on the website, I had the opportunity a year ago and passed. Its a personal regret of mine to not have worked with Mark on the website and help him along when I could have. He appreciated my early enthusiasm but I got busy with work and life. You know how it is.

Anyway all that aside, Mark ran a very tight ship and learning his processes will take some time. If you do see the website go down in the meantime know that its probably me on the other end trying to resolve certificate issue :)
Joseph Dickson | www.linuxbookpro.com | www.joseph-dickson.com | https://twitter.com/joe4ska - "I also enjoy Debian Stable, Pop!_OS, and Trisquel."

Offline stevesveryown

  • Trusted User
  • Member
  • *****
  • Posts: 423
  • Karma: 173
  • Peppermint Fan
    • View Profile
    • stevesveryown
  • Peppermint version(s): Peppermint 10
Re: A note on Sunday's server downtime
« Reply #5 on: March 02, 2020, 12:57:07 pm »
Thanks for your continued support @stevesveryown. Mark was very good at what he did in this project and no one person can replace such a dedicated project lead. We didn't have a second person on the website, I had the opportunity a year ago and passed. Its a personal regret of mine to not have worked with Mark on the website and help him along when I could have. He appreciated my early enthusiasm but I got busy with work and life. You know how it is.

Anyway all that aside, Mark ran a very tight ship and learning his processes will take some time. If you do see the website go down in the meantime know that its probably me on the other end trying to resolve certificate issue :)

With the support and encouragement from Mark over the last 2 years how can I not continue support for this project?

Offline stevesveryown

  • Trusted User
  • Member
  • *****
  • Posts: 423
  • Karma: 173
  • Peppermint Fan
    • View Profile
    • stevesveryown
  • Peppermint version(s): Peppermint 10
Re: A note on Sunday's server downtime
« Reply #6 on: March 02, 2020, 12:59:12 pm »
Also some big proponents of Peppermint in the YouTube community have already jumped ship.  I will be here and patiently await to see what happens.

Might you alert us to who is jumping ship so the rest of #teampeppermint may make contact @stevesveryown? 

 ;)

Sure!

Offline rayzer

  • Trusted User
  • Member
  • *****
  • Posts: 439
  • Karma: 181
  • Graphics n Stuff
    • View Profile
  • Peppermint version(s): 10 Respin
Re: A note on Sunday's server downtime
« Reply #7 on: March 02, 2020, 01:56:32 pm »
i expected all these issues to arise as ive seen it happen on another forum i used to frequent. the owner passed and he didnt pass any of the forum portal or hosting logins onto any of his staff as he was kinda paranoid about the security of it (long story). the forum simply just went offline as time went on and nobody could do anything about it. Thankfully the staff here are on top of things even if it is a difficult time.
For wallpapers ive made you can download them here

Offline spence

  • Administrator
  • Hero
  • *****
  • Posts: 3031
  • Karma: 345
  • peppermint user since 2010
    • View Profile
  • Peppermint version(s): Peppermint 10 Respin
Re: A note on Sunday's server downtime
« Reply #8 on: March 02, 2020, 03:19:52 pm »
I seem to recollect a discussion about that @rayzer, mind refreshing my memory about which distro that was?
spence
PeppermintOS 10 Respin currently installed  on:
'16 Antec Aria rebuild
& '18 Asus VivoBook


Do not despair, grasshopper...
    with patience all will be revealed...
       Through pain, enlightenment will come.

Offline rayzer

  • Trusted User
  • Member
  • *****
  • Posts: 439
  • Karma: 181
  • Graphics n Stuff
    • View Profile
  • Peppermint version(s): 10 Respin
Re: A note on Sunday's server downtime
« Reply #9 on: March 03, 2020, 10:27:25 am »
was a music forum spence, nothing to do with linux
For wallpapers ive made you can download them here

Offline josephd

  • Administrator
  • Member
  • *****
  • Posts: 763
  • Karma: 270
  • Peppermint OS Website Admin
    • View Profile
    • LinuxBookPro
  • Peppermint version(s): Peppermint 10
Re: A note on Sunday's server downtime [solved]
« Reply #10 on: March 03, 2020, 10:10:16 pm »
The security certificate problem has been solved. Thank you everyone for your patience while I worked on this issue.

Joseph Dickson | www.linuxbookpro.com | www.joseph-dickson.com | https://twitter.com/joe4ska - "I also enjoy Debian Stable, Pop!_OS, and Trisquel."

Online VinDSL

  • Administrator
  • Hero
  • *****
  • Posts: 5902
  • Karma: 1160
  • Team Peppermint
    • View Profile
  • Peppermint version(s): Developmental Builds
Re: A note on Sunday's server downtime [solved]
« Reply #11 on: March 04, 2020, 05:55:17 am »
The security certificate problem has been solved. Thank you everyone for your patience while I worked on this issue.

We must have been on the site at the same time. The cert was still borked at 10:00 PM, so I went about doing some daily maintenance.

About 10:15 I started getting 404s. I figured you were working on it.

Then, around 10:30 the connection(s) returned, and the SSL was working.

I was gonna shoot you a congrats via Proton, but we've been-there-done-it before, just to have it crash a few minutes later. I thought... why tempt fate!?  LoL

Anyway, good job, Detective Joe. Onward and upward    8)
« Last Edit: March 04, 2020, 06:01:40 am by VinDSL »

Offline stevesveryown

  • Trusted User
  • Member
  • *****
  • Posts: 423
  • Karma: 173
  • Peppermint Fan
    • View Profile
    • stevesveryown
  • Peppermint version(s): Peppermint 10
Re: A note on Sunday's server downtime [solved]
« Reply #12 on: March 04, 2020, 06:30:48 am »
The security certificate problem has been solved. Thank you everyone for your patience while I worked on this issue.



Awesome, Joseph!

Offline TheAkantor

  • Jr. Member
  • **
  • Posts: 25
  • Karma: 8
  • New Forum User
    • View Profile
  • Peppermint version(s): Peppermint 10 Respin
Re: A note on Sunday's server downtime
« Reply #13 on: March 04, 2020, 06:42:28 am »
Nice work Josephd , next time you go drink a beer put it in my account  8)

Offline jlschwartz

  • Trusted User
  • Member
  • *****
  • Posts: 402
  • Karma: 169
  • Forum User
    • View Profile
  • Peppermint version(s): Peppermint 10 - 64bit
Re: A note on Sunday's server downtime
« Reply #14 on: March 04, 2020, 06:54:38 am »
Thanks Josephd.  :)