Update - We have successfully finished the repair action of our NoSQL database and have included a new baremetal node in our cluster. We need to run a new Join + repair action with this new node to finish our migration. We expect minor latencies during this process that should not impact your daily Ubidots usage. We expect to finish this process next week.
May 4, 13:02 UTC
Update - We have identified a pattern where our time-series database cluster experiences a 15-minute degradation at 11:00, 12:00, 1:00, and 2:00 GMT-5 every day. This degradation is approximately 1/10th of the capacity (the SAN throughput goes from 200MB/s to 20MB/s), causing some dashboard widgets to display an error message during such times. The degradation slowly fades after a few minutes of such window, once the pending requests are processed.

About the SAN issues, our service provider is suggesting a server reboot will likely solve this, as it would discard OS-related issues. Unfortunately, this is not possible at this time because, given the initial degradation, we spun up a new server to increase overall cluster performance, and such a "cluster joining process" takes a few days, given the size of our database.

We are expecting that the new cluster node will successfully join within a week, after which we will be able to gradually reboot each cluster node to a new, fresh start.
Apr 29, 12:31 UTC
Update - We are still working on joining a new node to our cluster in order to alleviate the workload of our database access, unfortunately, this action is taking more than expected. We have detected additional SAN latency at our bare-metal infrastructure provider, and we are working with them in order to improve the write/read rate in our HDDs, this should improve the Database speed access. In the mean time, some users may experience latency when trying to access their data from the Dashboard or Variable view at certain times of the day. Synthetic variables calculation is expected to take longer than usually.
There are no issues with data ingestion.
Apr 27, 11:55 UTC
Update - We are experiencing latency for data retrieval for some requests, it is not a general issue but may affect some users that attempt to load dashboards or variables. The problem is not general and is raised just in certain moments during the daily Ubidots usage. We are working in the deployment of new cluster instances to solve it.
Apr 22, 14:36 UTC
Update - We have deployed a repair action to our cluster, this may derivate latency to our database for data retrieval and synthetic variables calculation. We do not expect any outage in our core systems, and this expected latency should not affect your daily Ubidots usage. This action may take the whole week to be finished
Apr 20, 12:46 UTC
Monitoring - We are experiencing latency during query execution at our database cluster, due to this, synthetic variables that usually take 1 minute to be calculated may take from 1:30 to 2:30, this behavior is extended to greater compute time variables. Dashboards and variables and device view may also experience a delay to be loaded. This delay should not be critical for our customer's applications.

We are working to solve this issues as soon as possible
Apr 12, 22:32 UTC
Functions Operational
90 days ago
99.6 % uptime
Today
America Operational
90 days ago
99.99 % uptime
Today
Login Apps Toronto ? Operational
90 days ago
99.98 % uptime
Today
HTTP Post Toronto ? Operational
90 days ago
99.99 % uptime
Today
TCP Toronto ? Operational
90 days ago
99.98 % uptime
Today
MQTT Publish Toronto ? Operational
90 days ago
99.99 % uptime
Today
MQTT Subscribe Toronto ? Operational
90 days ago
99.99 % uptime
Today
Events Engine Toronto ? Operational
90 days ago
99.99 % uptime
Today
UDP Toronto ? Operational
90 days ago
100.0 % uptime
Today
Ubidots Australia (Private deployment) ? Operational
90 days ago
99.98 % uptime
Today
Login Apps Sydney ? Operational
90 days ago
99.97 % uptime
Today
MQTT Publish Sydney ? Operational
90 days ago
99.98 % uptime
Today
TCP Sydney ? Operational
90 days ago
99.99 % uptime
Today
Events Engine Sydney ? Operational
90 days ago
99.98 % uptime
Today
HTTP ? Operational
90 days ago
99.99 % uptime
Today
SMS ? Operational
Twilio REST API Operational
Twilio SMS, Latin America Operational
Twilio SMS, Europe Operational
Voice ? Operational
Twilio TwiML Operational
Twilio Voice, Latin America Operational
Twilio Voice, Europe Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
REST API
Fetching
Past Incidents
May 16, 2021

No incidents reported today.

May 15, 2021

No incidents reported.

May 14, 2021

No incidents reported.

May 13, 2021

No incidents reported.

May 12, 2021

No incidents reported.

May 11, 2021

No incidents reported.

May 10, 2021

No incidents reported.

May 9, 2021

No incidents reported.

May 8, 2021

No incidents reported.

May 7, 2021

No incidents reported.

May 6, 2021

No incidents reported.

May 5, 2021

No incidents reported.

May 4, 2021

Unresolved incident: DataBase latency.

May 3, 2021

No incidents reported.

May 2, 2021

No incidents reported.