Quickpay - Notice history

API - Operational

100% - uptime
Jul 2022 · 100.0%Aug · 100.0%Sep · 100.0%
Jul 2022
Aug 2022
Sep 2022

Payment - Operational

100% - uptime
Jul 2022 · 100.0%Aug · 100.0%Sep · 100.0%
Jul 2022
Aug 2022
Sep 2022

manage.quickpay.net - Operational

100% - uptime
Jul 2022 · 100.0%Aug · 100.0%Sep · 100.0%
Jul 2022
Aug 2022
Sep 2022

3rd party services - Operational

100% - uptime
Jul 2022 · 100.0%Aug · 100.0%Sep · 100.0%
Jul 2022
Aug 2022
Sep 2022

Quickpay Support Mail - Operational

100% - uptime
Jul 2022 · 100.0%Aug · 100.0%Sep · 100.0%
Jul 2022
Aug 2022
Sep 2022

Notice history

Sep 2022

Problems with our online systems
  • Resolved
    Update

    Post mortem

    On the night between Friday September 2, 2022 and Saturday September 3, 2022, we experienced a system outage. The outage affected all payment methods from 00.15 > 01.15 CEST. From 01.15 > 11.30 CEST we still experienced downtime for payments made via Vipps, Swish & Klarna.

    The outage was caused by a hardware failure on our database.

    Our systems are designed with failover in mind, with a backup database always available and up to date with the primary. The system is designed to automatically switch to the backup database, and use this as the primary in case of hardware failures.

    Unfortunately this did not happen friday. As a result, all transactions were refused.

    Our operation team was quickly alerted, but the error that caused the systems to not automatically switch, also made the manual fix more difficult . At around 01.15, the team were able to restart all payment services running on the new database, and transactions were again processed.

    How to make sure this does not happen again

    Hardware failures are part of the game of self-hosted systems. Quickpay are in the process of migrating our systems to being cloud hosted, which will make sure that such hardware failures do not happen again.

    Until this migration is completed, we cannot avoid hardware failures, but we can make sure that the services will automatically switch to the backup database.

    We are already working on fixing the problems, and making sure that if such a problem should happen again, the systems are able to correct themselves quickly, without anyone noticing.

    Why was nothing announced on status.quickpay.net

    We are sorry that nothing was announced at the time of the outage. When we have an outage that affects all payments our first priority is to fix the issue at hand. We have a status-page service provider that allows us to make default announcements which did not work for various reasons, this will also be fixed to make sure that we communicate as quickly as possible in the future."

  • Resolved
    Update

    Additionally payments with some payment methods have been affected longer.

    This problem is also fixed now, and we are investigating and monitoring the situation

  • Resolved
    Resolved

    Sep 3 from 00:51 to 01:15 CEST, we experienced problems with the payment systems.

    All transactions were refused.

    The problem was fixed by our operations team. We will provide a more detailed update later this week.

    Best regards, Quickpay

Aug 2022

Shopify update
  • Resolved
    Resolved

    English version below

    Alle Shopify butikker med en 3. parts betalingsløsning, som Quickpay, har skullet migreres over på et nyt betalings API hos Shopify i løbet af sommeren med en deadline per 01/08/2022.

    Med migreringen til den nye app har mange Quickpay kunder desværre oplevet problemer med betalinger. Primært, men ikke isoleret, til forkert status på betalinger. Disse fejl har været sporadiske, og har været i visse betalingsflows og mere eller mindre konsistente på bestemte butikker.

    Selvom det har haft høj prioritet hos os fra start, har skal der ikke være nogen tvivl om at det har taget for lang tid at løse disse problemer.

    Mine kollegaer i supporten står forrest i skudlinjen, når kunderne har spørgsmål og bekymringer. Deres mailboks har været fyldt, og telefonerne har været glødende - på grund af mangler i vores Shopify app. Det har du som kunde måske også fornemmet, når du har været i dialog med dem. Jeg vil gerne beklage, hvis vores service i denne periode ikke har levet op til vores normale standarder.

    Kort status på appen i dag

    Vi har løbende over den seneste tid rullet nye opdateringer ud, der har fjernet dele af problemerne. De fleste problemer skulle være løst med opdateringer sendt ud onsdag og fredag i uge 32.

    Skulle I stadigt være berørte af problemerne, skal I endeligt tage fat i vores support igen. Jo flere eksempler vi får, jo hurtigere kan vi løse dem. Derudover kan I følge med på https://quickpay.net/integrations/shopify, hvor vi løbende vil opdatere.

    Næste skridt af at få flere af de enkelte betalingsapps godkendt af Shopify. Herunder Anyday, Vipps og Swish. Denne process er i store dele ude af vores hænder, og op til en godkendelsesprocess hos Shopify. Men det er vores forventning, at de løbende vil komme over de næste uger.

    Jeg er klar over, at de mange fejl og mangler i vores Shopify app har givet anledning til frustrationer og bekymringer i din hverdag, og det undskylder vi for.

    På vegne af mig selv og mine kollegaer i Quickpay ønsker vi dig en rigtig god sommer.

    Med venlig hilsen

    Mark Eskelund, Quickpay

    English version

    All Shopify merchants with a 3rd party payment gateway, like Quickpay, have been required to migrate to a new payment API at Shopify during the summer, with a deadline of August 1st 2022.

    With the migration to the new app, many Quickpay merchants have unfortunately experienced problems with their payments. Primarily, but not isolated to, that the status of orders in Shopify have for some payments, been incorrect. For some merchants these issues have been sporadic, for others consistent, and some merchants have not had any problems.

    Even though it has had high priority from the beginning, there is no doubt that solving these problems have taken too long.

    My colleagues in customer support are in the frontlines when merchants have questions or concerns. Their mailboxes have been full, and the phones have been ringing non stop - due to the problems with our Shopify app. As a merchant you might have sensed this, when you have been in touch with them. I would like to apologize, if our service during this process has not met the standards you would usually expect from us.

    Short status on the app today

    We have continuously rolled out updates for the app that have fixed issues. The majority of the problems should be solved with the latest updates that were rolled out on Wednesday and Friday in week 32.

    Should you still be affected by these problems, do not hesitate to contact our support again. The more examples we get, the quicker we can solve them. You can follow along on https://quickpay.net/integrations/shopify, where we will keep the article updated.

    Next step from us is to get more of the individual payment methods approved by Shopify. Hereunder Anyday, Vipps and Swish. This process is mainly out of our hands, and left to the Shopify approval process. However we expect to roll these out over the following weeks.

    I am fully aware that the problems have caused great frustrations and concerns for your daily work, and for that we apologize.

    On behalf of myself and my colleagues in Quickpay we would like to wish for a great summer.

    Best regards

    Mark Eskelund, Quickpay

Jul 2022 to Sep 2022

Next