Skip to content

Commit

Permalink
Synchronized build
Browse files Browse the repository at this point in the history
  • Loading branch information
@lektor-bot
lektor-bot committed Oct 19, 2023
1 parent 26056a5 commit 89e7993
Show file tree
Hide file tree
Showing 14 changed files with 894 additions and 631 deletions.
214 changes: 214 additions & 0 deletions 2023/10/19/2023-10-19-upgrade-issues/index.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8">

<meta property="og:image" content="https://blog.zenodo.org/static/img/logos/zenodo-gradient-1000.png" />


<meta property="og:title" content="Zenodo upgrade issues" />


<meta property="og:type" content="article" />



<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:site" content="@zenodo_org" />
<meta name="twitter:creator" content="@zenodo_org" />

<meta property="og:title" content="Zenodo upgrade issues" />



<meta name="twitter:description" content="Zenodo is a free and open digital archive built by CERN and OpenAIRE, enabling researchers to share and preserve research output in any size, format and from all fields of research." />

<meta name="twitter:image" content="https://blog.zenodo.org/static/img/logos/zenodo-gradient-1000.png" />
<link rel="stylesheet" href="../../../../static/zenodo.css">
<link rel="stylesheet" href="../../../../static/style.css">
<link rel="stylesheet" href="../../../../static/font-awesome/css/font-awesome.min.css">
<link href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700,100,italic" rel="stylesheet">
<title>Zenodo</title>
<script src="https://code.jquery.com/jquery-3.1.1.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
</head>
<body>
<header>
<nav class="navbar navbar-default navbar-static-top">
<div class="container">
<div class="navbar-header">
<a href="https://zenodo.org"><img class="navbar-brand" src="/static/img/zenodo.svg" alt="Zenodo" /></a>
<span class="logo-divider"></span>
<span class="header-title" aria-label="header-title">Blog</span>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul class="nav navbar-nav navbar-right toplist">

<li><a href="https://about.zenodo.org">About</a></li>

<li><a href="https://blog.zenodo.org">Blog</a></li>

<li><a href="https://help.zenodo.org">Help</a></li>

<li><a href="https://about.zenodo.org/projects">Projects</a></li>

<li><a href="https://developers.zenodo.org">Developers</a></li>

</ul>
</div>
</div>
</nav>
</header>
<div class="communities">


<div class="container body-container">
<div class="row">
<div class="col-md-12">



<h1>Zenodo upgrade issues</h1>

<p class="meta">
by



<img class="img-rounded" width=18 height=18 src="../../../../static/img/authors/lnielsen.jpg" />



Lars Holm Nielsen,

on October 19, 2023
</p>
<hr />
<div class="body"><p>As many of you noticed, the Zenodo upgrade last Friday didn’t go as smoothly as hoped. Afterwards Zenodo was sluggish and file uploads were painfully slow, some features didn’t quite work as expected. We’ve been working flat-out behind the scenes to steer Zenodo out of these stormy waters. We’d like to keep you updated on the issues we had, what we solved so far, and what we are still working on.</p>
<p>The upgrade involved orchestration of three components; Zenodo’s refactored software platform, the data, and the technical infrastructure:</p>
<ol>
<li>We’d developed the refactored software platform in collaboration with partners around the world, 6 of which already had production instances running on the new code (*), giving confidence, but we none the less tested extensively on the full feature set used on Zenodo. We’d been smoothing off rough edges, and were prepared to rapidly address more that you might discover after release.</li>
<li>As custodians of your data, for months we meticulously exercised the process of migration to the new system to ensure every bit made it reliably, and also have multiple backups as safety and for verification.</li>
<li>We’d revamped the technical infrastructure to better serve the continued scaling demanded by Zenodo’s ceaseless growth, by using components heavily used in our other front-line services, which none the less we also stress/performance tested in the preparations.</li>
</ol>
<p>The main issue after the upgrade was immediately evident that file upload/download was painfully slow (1GB file taking 1 hour instead of 20 seconds). We worked the weekend to finally discover that circumventing the front-facing load balancer alleviated the problem, and on removing it we restored Zenodo’s expected data performance (the root cause is still being investigated by the infrastructure experts).</p>
<p>We also had indexing issues with records which meant they didn't show up despite the data having been migrated, and despite having had successful test migration runs. We have almost finished to resolve this issue and expect the main issues to have been solved by Monday.</p>
<p>We want to thank you for all the incredible support and understanding that we have been receiving even under such difficulties. We’ve not ignored the functionality hiccups you’ve been reporting, we already solved some in parallel to the performance debugging. Please rest assured that we continue to work relentlessly to address all your support requests and feedback, and to address all outstanding issues in a timely manner to help you publish/access your content.</p>
<p>We apologise for all the inconvenience it has caused. We hope once we're over the hiccups the new Zenodo will serve you and your communities well!</p>
<p>Final lesson learnt: Friday 13th[**] might not have been a good day for a major release after all.</p>
<p>(*)</p>
<ul>
<li>Northwestern Uni: <a href="https://prism.northwestern.edu/">https://prism.northwestern.edu/</a></li>
<li>NYU: <a href="https://ultraviolet.library.nyu.edu/">https://ultraviolet.library.nyu.edu/</a></li>
<li>GEO Knowledge Hub: <a href="https://gkhub.earthobservations.org/">https://gkhub.earthobservations.org/</a></li>
<li>Caltech Library: <a href="https://data.caltech.edu/">https://data.caltech.edu/</a></li>
<li>TUGraz: <a href="https://repository.tugraz.at/">https://repository.tugraz.at/</a></li>
<li>TUWien: <a href="https://researchdata.tuwien.ac.at/">https://researchdata.tuwien.ac.at/</a></li>
</ul>
<p>(**) We chose a Friday as Zenodo has considerable less traffic during weekends and thus as risk mitigation measure we would impact less users if there were problems (as there unfortunately turned out to be).</p>
</div>


</div>
</div>
</div>

</div>
<footer class="footer">
<div class="menu-wrapper">
<div class="container">
<div class="row footer-menu">
<div class="col-xs-12 col-md-8">
<div class="row">
<div class="col-xs-2 col-md-2">
<h5>About</h5>
<ul class="list-unstyled">
<li><a href="http://about.zenodo.org">About</a></li>
<li><a href="http://about.zenodo.org/policies/">Policies</a></li>
<li><a href="http://about.zenodo.org/infrastructure/">Infrastructure</a></li>
<li><a href="http://about.zenodo.org/principles/">Principles</a></li>
<li><a href="http://about.zenodo.org/roadmap/">Roadmap</a></li>
<li><a href="http://about.zenodo.org/projects/">Projects</a></li>
<li><a href="http://about.zenodo.org/contact/">Contract</a></li>
</ul>
</div>
<div class="col-xs-2 col-md-2">
<h5>Blog</h5>
<ul class="list-unstyled">
<li><a href="https://blog.zenodo.org">Blog</a></li>
</ul>
</div>
<div class="col-xs-2 col-md-2">
<h5>Help</h5>
<ul class="list-unstyled">
<li><a href="https://help.zenodo.org">Overview</a></li>
<li><a href="https://help.zenodo.org/faq/">FAQ</a></li>
<li><a href="https://help.zenodo.org/guides/">Guides</a></li>
<li><a href="https://zenodo.org/support">Support</a></li>
</div>
<div class="col-xs-2 col-md-2">
<h5>Developers</h5>
<ul class="list-unstyled">
<li><a href="https://developers.zenodo.org">REST API</a></li>
<li><a href="https://developers.zenodo.org#oai-pmh">OAI-PMH</a></li>
</ul>
</div>
<div class="col-xs-2 col-md-2">
<h5>Contribute</h5>
<ul class="list-unstyled">
<li><a href="https://github.com/zenodo/zenodo"><i class="fa fa-external-link"></i> GitHub</a></li>
<li><a href="https://zenodo.org/donate"><i class="fa fa-external-link"></i> Donate</a></li>
</ul>
</div>
</div>
</div>
<div class="col-xs-12 col-md-4">
<div class="pull-right-md text-center-sm text-center-xs">
<h5>Funded by</h5>
<ul class="list-inline">
<li><a href="https://home.cern"><img src="/static/img/cern.png" width="60" height="60" /></a></li>
<li><a href="https://www.openaire.eu"><img src="/static/img/openaire.png" width="80"/></a></li>
<li><a href="https://ec.europa.eu/programmes/horizon2020/"><img src="/static/img/eu.png" width="88" height="60" /></a></li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="col-xs-12 col-sm-6 col-sm-push-6">
<div class="pull-right-sm text-center-xs">
<ul class="list-inline">
<li><a href="https://stats.uptimerobot.com/vlYOVuWgM">Status</a></li>
<li><a href="https://about.zenodo.org/privacy-policy">Privacy policy</a></li>
<li><a href="https://about.zenodo.org/terms">Terms of Use</a></li>
<li><a href="https://about.zenodo.org/contact">Support</a></li>
</ul>
</div>
</div>
<div class="col-xs-12 col-sm-6 col-sm-pull-6 text-center-xs">
<p><a title="Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License." rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons Licence" height="20" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a>&nbsp; Powered by <a href="https://home.cern/science/computing/data-centre">CERN Data Centre</a> &amp; <a href="http://inveniosoftware.org">Invenio</a>.</p>
</div>
</div>
</div>
</footer>

<!-- Matomo -->
<script>
var _paq = window._paq = window._paq || [];
/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
var u="https://webanalytics.web.cern.ch/";
_paq.push(['setTrackerUrl', u+'matomo.php']);
_paq.push(['setSiteId', '362']);
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
})();
</script>
<!-- End Matomo Code -->

</body>
</html>
4 changes: 4 additions & 0 deletions all/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,10 @@ <h1>All posts</h1>

<ul class="blog-index">

<li>
<a href="../2023/10/19/2023-10-19-upgrade-issues/">Zenodo upgrade issues</a> — by Lars Holm Nielsen on Oct 19, 2023
</li>

<li>
<a href="../2023/10/13/2023-10-13-zenodo-rdm/">Zenodo launched on next generation platform - InvenioRDM 🚀</a> — by Lars Holm Nielsen on Oct 13, 2023
</li>
Expand Down
Loading

0 comments on commit 89e7993

Please sign in to comment.