tag:status.eventpower.com,2005:/historyeventPower Status - Incident History2024-03-29T01:24:00-04:00eventPowertag:status.eventpower.com,2005:Incident/197527552024-01-20T23:42:07-05:002024-01-20T23:42:08-05:00Scheduled Maintenance<p><small>Jan <var data-var='date'>20</var>, <var data-var='time'>23:42</var> EST</small><br><strong>Completed</strong> - Scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'>20</var>, <var data-var='time'>22:00</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>09:11</var> EST</small><br><strong>Scheduled</strong> - Systems will be undergoing scheduled maintenance during this time. We expect short periods of unavailability during this window. <br /><br />Please contact your eventPower contact or devs@eventpower.com with any questions or concerns.</p>tag:status.eventpower.com,2005:Incident/182246532023-08-26T23:49:57-04:002023-08-26T23:49:57-04:00Scheduled Maintenance Window<p><small>Aug <var data-var='date'>26</var>, <var data-var='time'>23:49</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>26</var>, <var data-var='time'>22:01</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>15:27</var> EDT</small><br><strong>Scheduled</strong> - We have scheduled a maintenance window for eventPower Tools beginning at 10pm EDT / 7pm PDT on Saturday 8/26. During this window we anticipate brief outages of all our tools and websites.<br /><br />We will be performing maintenance minor upgrades to ensure we stay current with security, server, and performance throughout our server fleet.<br /><br />Please reach out to your eventPower contact or devs@eventpower.com with any questions or concerns.</p>tag:status.eventpower.com,2005:Incident/178029442023-07-09T04:25:17-04:002023-07-09T04:25:17-04:00Services offline<p><small>Jul <var data-var='date'> 9</var>, <var data-var='time'>04:25</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved and systems are running as expected.<br /><br />We take downtime seriously and have addressed some of the underlying issues related to this and other downtime events. We will provide a postmortem this coming week explaining the changes the team is implementing to reduce or eliminate this unavailability in the future.</p><p><small>Jul <var data-var='date'> 9</var>, <var data-var='time'>00:00</var> EDT</small><br><strong>Identified</strong> - We've identified the issue with the database having gone through an automated update process and not fully recovering. We're working with our database team to determine the cause and get the systems back up and running. We anticipate this to be complete shortly.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>22:44</var> EDT</small><br><strong>Investigating</strong> - We are continuing to investigate the database issue.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>22:40</var> EDT</small><br><strong>Monitoring</strong> - Looks like we may have jumped the gun on status notifications. We are seeing services fully active and are continuing to monitor our services.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>22:39</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>22:39</var> EDT</small><br><strong>Investigating</strong> - We are experiencing some issues with our database cluster and are investigating the issue.</p>tag:status.eventpower.com,2005:Incident/175264352023-06-09T17:47:53-04:002023-06-09T17:47:53-04:00Tools Down<p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>17:47</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved. Database entered read_only mode and needed to have this flag turned off manually. The cause will continue to be investigated.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>17:24</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating the issue.</p>tag:status.eventpower.com,2005:Incident/174234542023-05-30T15:37:39-04:002023-05-30T15:37:39-04:00Some icons not loading<p><small>May <var data-var='date'>30</var>, <var data-var='time'>15:37</var> EDT</small><br><strong>Resolved</strong> - We believe this issue has been resolved and will continue to monitor over the next few hours.</p><p><small>May <var data-var='date'>30</var>, <var data-var='time'>13:37</var> EDT</small><br><strong>Identified</strong> - We have received reports that icons in eventPower's tools are not loading properly. The provider we work with for this capability is investigating the issue.<br /><br />This should not directly affect any functionality, though may make some screens look different than you're used to. We'll provide an update shortly when we receive more information.</p>tag:status.eventpower.com,2005:Incident/163085352023-02-28T19:31:01-05:002023-02-28T19:31:01-05:00Slow response and errors<p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>19:31</var> EST</small><br><strong>Resolved</strong> - As all services have been running as expected over the past hour or so, we are considering this issue resolved. We appreciate everyone's patience during this time.<br /><br />The team will be performing a post mortem of this incident and will publish it later this week on our status page at https://status.eventpower.com</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>18:10</var> EST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are seeing services being restored at this time. We're continuing to monitor the situation and will keep the team updated here.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>17:17</var> EST</small><br><strong>Update</strong> - We have identified the cause of the issues to be a failed database cluster. We are continuing to work diligently to resolve the issue.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>16:51</var> EST</small><br><strong>Update</strong> - We are continuing to investigate this issue with slow performance and server errors. <br />Note that event websites are fully available.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>16:24</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing errors on our primary services. Our team is currently looking into the issues.</p>tag:status.eventpower.com,2005:Incident/159554332023-01-27T16:55:51-05:002023-01-27T16:55:51-05:00500 Errors being reported<p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>16:55</var> EST</small><br><strong>Resolved</strong> - All systems are back running as expected and we are considering this resolved. <br /><br />We apologize for any inconvenience these couple minutes of unavailability may have caused and we will be looking into the root cause.</p><p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>16:40</var> EST</small><br><strong>Monitoring</strong> - We’re continuing to monitor.</p><p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>16:37</var> EST</small><br><strong>Identified</strong> - We believe we have isolated the issue to a failed database node. We are seeing services recovering and are monitoring at this time.</p><p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>16:33</var> EST</small><br><strong>Investigating</strong> - We are aware of users receiving errors across all eventPower systems. We are investigating the errors now and will provide an update shortly.</p>tag:status.eventpower.com,2005:Incident/138742032022-11-28T12:57:38-05:002022-11-28T12:57:38-05:00Monitoring reported errors<p><small>Nov <var data-var='date'>28</var>, <var data-var='time'>12:57</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>28</var>, <var data-var='time'>12:29</var> EST</small><br><strong>Monitoring</strong> - We were experiencing problems with our load-balanced database system which did not effectively handle the hand-off correctly. We are satisfied that the current issue has been resolved.<br /><br />We will be further evaluating the root cause of the problems we experienced today and will provide additional details as we have them.</p><p><small>Nov <var data-var='date'>28</var>, <var data-var='time'>11:59</var> EST</small><br><strong>Investigating</strong> - We're monitoring reports of some loading errors throughout our server fleet. We're seeing all services are currently available, but we are working with our hosting providers at this time.</p>tag:status.eventpower.com,2005:Incident/110476432022-09-03T17:57:20-04:002022-09-03T17:57:20-04:00Issues accessing sites<p><small>Sep <var data-var='date'> 3</var>, <var data-var='time'>17:57</var> EDT</small><br><strong>Resolved</strong> - We've confirmed all services are running as expected.</p><p><small>Sep <var data-var='date'> 3</var>, <var data-var='time'>17:41</var> EDT</small><br><strong>Monitoring</strong> - We've successfully restored all services and are monitoring all processes. <br /><br />Thank you for your patience during this unexpected extended outage. We will be reviewing and performing a full post-mortem of the incident and will share relevant details.</p><p><small>Sep <var data-var='date'> 3</var>, <var data-var='time'>16:40</var> EDT</small><br><strong>Update</strong> - Our team is continuing to work on the issue. We're currently working two directions to resolve the failed database cluster. We anticipate a resolution shortly.</p><p><small>Sep <var data-var='date'> 3</var>, <var data-var='time'>15:52</var> EDT</small><br><strong>Update</strong> - We are continuing to work with our hosting providers to work through this issue. We apologize for any inconvenience this is causing and will continue to update this page as we work towards a resolution of this issue.</p><p><small>Sep <var data-var='date'> 3</var>, <var data-var='time'>13:02</var> EDT</small><br><strong>Identified</strong> - We’ve determined the database cluster entered a failed state. The team is working to resolve the issue and we anticipate a resolution within 30-60 minutes. We will keep this page updated with information.</p><p><small>Sep <var data-var='date'> 3</var>, <var data-var='time'>12:40</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating an issue where some users are reporting errors accessing eventPower services.</p>tag:status.eventpower.com,2005:Incident/108866342022-08-25T23:53:44-04:002022-08-25T23:53:44-04:00Scheduled Maintenance<p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>23:53</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance was completed successfully. Thank you for your patience.</p><p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>22:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>17:18</var> EDT</small><br><strong>Scheduled</strong> - Maintenance is scheduled beginning at 10pm ET on Thursday 8/25 and continuing until 1am. During this time we will be upgrading some of our back-end servers.<br /><br />We anticipate up to 30-60 minutes of planned downtime of our administrative tools, registration, and virtual platforms during this period. Websites will experience some shorter downtime periods during this maintenance period.<br /><br />Please contact your eventPower contact with any questions or concerns.</p>tag:status.eventpower.com,2005:Incident/109198912022-08-25T12:14:00-04:002022-08-25T12:14:00-04:00Errors Reported<p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>12:14</var> EDT</small><br><strong>Resolved</strong> - Thank you for your patience during the downtime we experienced this morning. The root issue was a configuration change that was staged in preparation for tonight's maintenance, which was inadvertently deployed early. Unfortunately due to the configuration change, it wasn't immediately obvious that it was deployed or caused the issue.</p><p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>10:28</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring.</p><p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>10:13</var> EDT</small><br><strong>Investigating</strong> - We have received reports that some eventPower services are unavailable. We are looking into it now.</p>tag:status.eventpower.com,2005:Incident/96806132022-03-31T19:11:24-04:002022-03-31T19:11:24-04:00Admin server experiencing connectivity issues<p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>19:11</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>19:07</var> EDT</small><br><strong>Monitoring</strong> - We believe the issue has been resolved though the team continues to look into the reported problems.</p><p><small>Mar <var data-var='date'>31</var>, <var data-var='time'>19:04</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating an issue with our admin servers that may cause errors when accessing these systems. The team is looking into the reported problem.</p>tag:status.eventpower.com,2005:Incident/94821212022-03-07T13:39:43-05:002022-03-07T13:56:30-05:00Reports of Heavy Load on admin servers<p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>13:39</var> EST</small><br><strong>Resolved</strong> - We've successfully determined the causes of this morning's issues and have put in place short-term resolutions to prevent it from happening again.<br /><br />The team will additionally be reviewing capacity planning and real-time load shedding for unexpected significant spikes.<br /><br />We understand that you entrust eventPower with your events and we strive to ensure we provide the best availability we can. Please reach out to your eventPower contact with any questions or concerns.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>12:33</var> EST</small><br><strong>Monitoring</strong> - It appears that the heavy load issue has been resolved and the team is continuing to monitor the situation.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>12:24</var> EST</small><br><strong>Identified</strong> - The team believes the high load issue has been identified and is being resolved. We're already seeing errors clearing, but will continue monitoring the situation.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>11:57</var> EST</small><br><strong>Investigating</strong> - Our monitoring tools are reporting errors while connecting to admin tools and websites. <br /><br />The team is investigating and will update shortly.</p>tag:status.eventpower.com,2005:Incident/88549752021-12-15T11:44:49-05:002021-12-15T11:44:49-05:00Customer Service Currently Unavailable<p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>11:44</var> EST</small><br><strong>Resolved</strong> - All our vendors are reporting resolution and our team has confirmed all services are working as expected. We are marking this incident as resolved, though will follow-up with our vendors to determine the cause of this issue.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>11:36</var> EST</small><br><strong>Monitoring</strong> - We have received notification from our vendors that the situation has been resolved. We will continue monitoring.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>11:00</var> EST</small><br><strong>Update</strong> - It appears that AWS is experiencing connectivity issues in some of their North American data centers. This seems to be affecting a number of our third-party tools including our customer service tools and virtual platform chat.<br /><br />The eventPower platform continues to be fully up and running for all our users, though we continue to monitor the situation.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>10:39</var> EST</small><br><strong>Investigating</strong> - We have had reports that our customer service platform, Zendesk, is currently experiencing connectivity errors. Our customer service team can not currently field incoming tickets or requests. <br /><br />Attendees/users can continue to send emails to our customer service as these will be processed when the systems come back online.<br /><br />We are working with our vendor to monitor the situation and will provide updates as we receive them.</p>tag:status.eventpower.com,2005:Incident/87762482021-12-06T20:00:01-05:002021-12-06T20:45:45-05:00Online Registration Errors<p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>20:00</var> EST</small><br><strong>Resolved</strong> - For a short period of time this evening, the online registration process was returning errors for some users. Once our team noticed the issue, it was quickly resolved and registration is functioning as expected for all users.<br /><br />We were able to track the problem back to an error in a minor feature update that was not caught by peer code review, automated integration testing, and not immediately noticed in the production environment. <br /><br />Reliability is of utmost importance to our team, as we know it's critical for you and your events. We are continually improving our processes to catch problems like this well before they make it into production systems, and will utilize this as an opportunity to further hone our testing process; and continue to improve our communication with our clients.</p>tag:status.eventpower.com,2005:Incident/86719332021-11-26T23:09:59-05:002021-11-26T23:09:59-05:00Maintenance Window (Friday 10pm-12)<p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>23:09</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>22:05</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>23</var>, <var data-var='time'>09:48</var> EST</small><br><strong>Scheduled</strong> - We have planned system maintenance from 10pm - 12 midnight EST on Friday for system upgrades. We anticipate up to 30 minutes of downtime during this maintenance window.<br /><br />Please contact your eventPower contact with any concerns.</p>tag:status.eventpower.com,2005:Incident/82366742021-10-15T10:36:17-04:002021-10-15T10:36:17-04:00Server Unavailability<p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>10:36</var> EDT</small><br><strong>Resolved</strong> - We have resolved the cause of this morning's issue and the team is reviewing the details of the root cause. All services are running reliably.<br /><br />We understand unavailability is difficult for all clients and users and our team strives to provide the best service possible to all our users. We will be following up with a post mortem from this event within the next day or two.</p><p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>10:04</var> EDT</small><br><strong>Identified</strong> - We've identified and resolved the main issue that caused the problems. Services and up and running and the team is working to ensure the stability is maintained. We'll continue to provide updates here.</p><p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>09:54</var> EDT</small><br><strong>Update</strong> - We've discovered the cause of the issues and are working to resolve them.</p><p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>09:51</var> EDT</small><br><strong>Update</strong> - The team is continuing to investigate the problems we are experiencing.</p><p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>09:40</var> EDT</small><br><strong>Investigating</strong> - We are troubleshooting an issue which is causing our tools to be unavailable and/or responding slowly. Our team is troubleshooting the issue and will provide an update shortly.</p>tag:status.eventpower.com,2005:Incident/81323382021-10-01T23:40:48-04:002021-10-01T23:40:48-04:00Maintenance Window (Friday 10:30pm)<p><small>Oct <var data-var='date'> 1</var>, <var data-var='time'>23:40</var> EDT</small><br><strong>Completed</strong> - This maintenance has been completed.</p><p><small>Oct <var data-var='date'> 1</var>, <var data-var='time'>22:30</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'> 1</var>, <var data-var='time'>15:16</var> EDT</small><br><strong>Scheduled</strong> - We have scheduled a maintenance window to perform urgent upgrades to certain software on our production servers. This maintenance is required to ensure secure and reliable communications between servers in our fleet.<br /><br />We anticipate brief, rolling downtime throughout all our services and servers of about 5-10 minutes throughout this maintenance window. <br /><br />Please contact your eventPower manager or devs@eventpower.com with any questions or concerns.</p>tag:status.eventpower.com,2005:Incident/76732962021-08-03T14:56:22-04:002021-08-03T14:56:22-04:00Networking issues on event app/virtual platform servers<p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>14:56</var> EDT</small><br><strong>Resolved</strong> - After a couple hours of no further issues and our monitoring tools reporting stable performance, we are going to mark this incident as resolved.<br /><br />Thank you and please reach out if you have any questions or are experiencing any problems.</p><p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>13:42</var> EDT</small><br><strong>Monitoring</strong> - A fix has been applied to resolve the connectivity issues we have been experiencing. We believe at this point the issues have been resolved but will continue to monitor.</p><p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>12:13</var> EDT</small><br><strong>Identified</strong> - Our network and hosting provider has confirmed connectivity issues within their datacenter resulting in issues with server interconnectivity. <br /><br />We are currently showing no issues on our admin (tools) servers and public-facing websites.<br />We are also showing about 40% of our virtual platform servers raising connectivity errors.<br /><br />We're continuing to work on this issue and determine if there are short-term rerouting that can be accomplished.</p><p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>11:58</var> EDT</small><br><strong>Update</strong> - We have been in contact with our networking and hosting providers and they are aware of connectivity issues within their network. <br /><br />Over the past few minutes, we have seen internal networking issues decreasing and services coming back online. We will continue investigating the situation and will provide additional details as they arise.</p><p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>11:53</var> EDT</small><br><strong>Investigating</strong> - We have received reports of some of our event app/virtual platform servers returning errors. We are looking into this and will report back shortly with more details.</p>tag:status.eventpower.com,2005:Incident/71963562021-06-08T07:11:06-04:002021-06-08T07:11:06-04:00Content may not fully load<p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>07:11</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>07:03</var> EDT</small><br><strong>Monitoring</strong> - Our partner has confirmed our network has been fully rerouted to functional CDNs worldwide (CloudFront, Akamai).<br /><br />Fastly CDN is continuing to report partial outages across their network, though this should no longer affect our systems. We will continue to monitor as the day progresses. <br /><br />It is interesting to note that a large swath of the internet has reported outages during this time including Reddit, Spotify, Twitch, Stack, gov.uk, Hulu, HBO Max, PayPal, Vimeo, Shopify, and news outlets CNN, the Guardian, the New York Times, BBC, Financial Times (Reported by ABC News).</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>06:47</var> EDT</small><br><strong>Update</strong> - Our partner is working to reroute traffic to functional CDNs. Additionally, the global CDN network is reporting that they have identified the issue and are implementing a fix.<br /><br />We are receiving reports from users indicating that the situation is improving.</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>06:41</var> EDT</small><br><strong>Identified</strong> - Our content delivery partner is experiencing a worldwide issue which is affecting the delivery of content throughout our site and tools. Images, videos, and other media may not load correctly during this incident.<br /><br />We are working with our provider to get services switched to another CDN network and will provide updates here.</p>tag:status.eventpower.com,2005:Incident/70767692021-05-24T14:30:00-04:002021-05-24T17:19:16-04:00Heavy Load on Event App (Virtual Platform) Servers<p><small>May <var data-var='date'>24</var>, <var data-var='time'>14:30</var> EDT</small><br><strong>Resolved</strong> - Beginning around 2:30pm ET, we began seeing abnormally heavy loads on the load-balanced event app servers. Initial analysis indicated a high number of requests from the eP Collaboration tool (Chime app). We were seeing users still able to access the virtual platform services, though receiving intermittent errors.<br /><br />We were able to implement a series of mitigations to address the short-term issue. Services were fully back up and running a bit before 4pm ET.<br /><br />We take availability seriously. As such, the team is currently reviewing both the cause and mitigations put in place to determine if they are sufficient and how best to alleviate these types of issues in the future. The Development/Engineering team will be providing updates as needed and will post a post-mortem within the next day or two.</p>tag:status.eventpower.com,2005:Incident/66584872021-04-04T00:00:32-04:002021-04-04T00:00:32-04:00Maintenance Window (Saturday 10pm-12pm EDT)<p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>00:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'> 3</var>, <var data-var='time'>22:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>16:32</var> EDT</small><br><strong>Scheduled</strong> - We have scheduled a maintenance window to proactively update some of our servers. We anticipate about 30 minutes of unavailability during this window. This will affect:<br />- Client/admin dashboards and reporting<br />- Registration/Speaker CFP/Exhibits tools<br />- Hosted websites<br />- eventPower website<br /><br />This maintenance window does not affect attendee access to the eventPower virtual platform, nor API functionality.<br /><br />Please contact your eventPower contact or devs@eventpower.com if you have any questions or concerns.</p>tag:status.eventpower.com,2005:Incident/64393192021-03-03T08:31:06-05:002021-03-03T08:31:06-05:00Network Instability<p><small>Mar <var data-var='date'> 3</var>, <var data-var='time'>08:31</var> EST</small><br><strong>Resolved</strong> - Our provider has confirmed they have successfully updated the switches. Since then, we haven't seen any anomalous behavior and are considering this issue resolved.</p><p><small>Mar <var data-var='date'> 2</var>, <var data-var='time'>22:22</var> EST</small><br><strong>Update</strong> - Our network provider is performing emergency maintenance to update the software on their pod switches tonight between midnight and 2am ET. They inform us that they anticipate no downtime during this. We will continue monitoring overnight and provide an update when we have additional information.<br /><br />At this time, we are seeing reasonable stability in all our systems.</p><p><small>Mar <var data-var='date'> 2</var>, <var data-var='time'>17:47</var> EST</small><br><strong>Monitoring</strong> - We haven't seen any additional network issues since we posted this incident. We are continuing to monitor and will update this incident as the day goes on.</p><p><small>Mar <var data-var='date'> 2</var>, <var data-var='time'>17:08</var> EST</small><br><strong>Identified</strong> - Our network provider is reporting intermittent packet loss which is affecting some of our app servers. We understand they have identified the source of the issue to a bug on a redundant set of switches in the datacenter. <br /><br />They are working on implementing emergency maintenance. <br /><br />At this time, we are seeing full service without problems - but we are continuing to monitor for short-term drops in service and will provide updates as we learn more.</p>tag:status.eventpower.com,2005:Incident/61853932021-02-03T14:50:43-05:002021-02-03T14:50:43-05:00Virtual Platform Chat - connectivity issues<p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>14:50</var> EST</small><br><strong>Resolved</strong> - Our chat provider is reporting that the issue has been fully resolved from their end. Please let our team know if you or your users experience any further problems or issues.</p><p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>14:49</var> EST</small><br><strong>Monitoring</strong> - Our team and users are reporting no issues over the past hour or so. We will continue monitoring the chat platform and provide an update if we continue to receive reports of any problems.</p><p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>14:05</var> EST</small><br><strong>Update</strong> - Our chat provider is continuing to work on this issue. We are seeing most chats going through, though there continues to be some slowness and infrequent errors.<br /><br />We'll continuing monitoring the situation and keep this page updated as we find out more information.</p><p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>12:59</var> EST</small><br><strong>Identified</strong> - Our chat partner is reporting elevated errors in their chat platform. This is causing some chat rooms and chat messages to fail to send within the eventPower virtual platform.<br /><br />All other eventPower services and features are running normally.<br /><br />We will continue to work with our chat provider and provide updates as we have them.</p>tag:status.eventpower.com,2005:Incident/61649292021-02-01T13:39:57-05:002021-02-01T13:39:57-05:00Server Unavailable<p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>13:39</var> EST</small><br><strong>Resolved</strong> - We are considering this issue as resolved. We will be reviewing the issues with our providers and will be providing a postmortem over the next few days with more details.<br /><br />Thank you for your patience during this time, we take unavailability seriously and strive to reduce and eliminate downtime whenever and however possible.</p><p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>11:37</var> EST</small><br><strong>Monitoring</strong> - All services are restored and running as usual. We are continuing to monitor.</p><p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>11:13</var> EST</small><br><strong>Identified</strong> - We are seeing all our services coming back online at this time. We're continuing to investigate to alleviate any continued issues. At this time, it seems that we are back up and running.</p><p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>10:59</var> EST</small><br><strong>Update</strong> - Our team is continuing to investigate the networking issue. In the interim, we are working plans for events that are live later today. Your project managers will be in touch with more detail for active events.</p><p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>10:31</var> EST</small><br><strong>Update</strong> - We are working with our network provider who has identified a potential issue in their network configuration. They are further investigating and we are expecting more information shortly.</p><p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>10:26</var> EST</small><br><strong>Investigating</strong> - We are experiencing issues with connecting to servers. We are investigating the issue at this time.</p>