mdl probes

_data/papers/2020.yml

@@ -0,0 +1,13 @@
+
+-
+  layout: paper
+  paper-type: inproceedings
+  selected: y
+  year: 2020
+  img: mdl_probes-min.png
+  title: "Information-Theoretic Probing with Minimum Description Length"
+  authors: "<u>Elena Voita</u> and Ivan Titov"
+  doc-url:
+  conf: no_conf.png
+  url: "https://arxiv.org/abs/2003.12298"
+

_site/Gemfile.lock

@@ -205,7 +205,7 @@ GEM
       jekyll-seo-tag (~> 2.1)
     minitest (5.11.3)
     multipart-post (2.1.1)
-    nokogiri (1.10.3)
+    nokogiri (1.10.8)
       mini_portile2 (~> 2.4.0)
     octokit (4.14.0)
       sawyer (~> 0.8.0, >= 0.5.3)

_site/feed.xml

@@ -2,12 +2,12 @@
 <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
   <channel>
     <title>Lena Voita</title>
-    <description>Here you will find details about my work.
+    <description>Hi! I'm Lena and here you will find what I do.
 </description>
     <link>http://localhost:4000/</link>
     <atom:link href="http://localhost:4000/feed.xml" rel="self" type="application/rss+xml" />
-    <pubDate>Tue, 26 Nov 2019 19:32:18 +0300</pubDate>
-    <lastBuildDate>Tue, 26 Nov 2019 19:32:18 +0300</lastBuildDate>
+    <pubDate>Mon, 30 Mar 2020 06:26:13 +0300</pubDate>
+    <lastBuildDate>Mon, 30 Mar 2020 06:26:13 +0300</lastBuildDate>
     <generator>Jekyll v3.8.5</generator>
 
   </channel>

_site/index.html

@@ -7,7 +7,7 @@
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
 
     <title>Lena Voita</title>
-    <meta name="description" content="Here you will find details about my work.
+    <meta name="description" content="Hi! I'm Lena and here you will find what I do.
 ">
 
     <link rel="stylesheet" href="/css/main.css">
@@ -60,6 +60,8 @@
 
 
 
+
+
           <a class="page-link" href="/posts.html">Blog</a>
 
 
@@ -94,22 +96,22 @@
 <!-- <a href= onMouseOver="document.readmore_1.src='/img/people/foxie.jpeg';" onMouseOut="document.readmore_1.src='/img/people/lena-min.png';">
 <img src="/img/people/lena-min.png" name="readmore_1" width=204px height=240px></a> -->
 
-<p>I am Elena (Lena) Voita, a <strong>Ph.D. candidate</strong> at the University of Amsterdam. Often I visit the <a href="https://www.ed.ac.uk/informatics">School of Informatics</a> at the University of Edinburgh, and I am part of the <a href="http://edinburghnlp.inf.ed.ac.uk">EdinburghNLP</a> group. I’m very happy to be supervised by <a href="http://ivan-titov.org">Ivan Titov</a> and <a href="http://homepages.inf.ed.ac.uk/rsennric/">Rico Sennrich</a>.</p>
+<p>I am Elena (Lena) Voita, a <strong>Ph.D. student</strong> at the University of Edinburgh and University of Amsterdam. I’m very happy to be supervised by <a href="http://ivan-titov.org">Ivan Titov</a> and <a href="http://homepages.inf.ed.ac.uk/rsennric/">Rico Sennrich</a>.</p>
 
-<p>I am also a <strong>research scientist</strong> at <a href="https://research.yandex.com">Yandex Research</a> working on <strong>Natural Language Processing</strong>. My research focuses on Neural Machine Translation, and I work closely with the <a href="https://translate.yandex.com">Yandex Translate</a> team<a onmouseover="document.fox.src='/img/people/foxie.jpeg';" onmouseout="document.fox.src='/img/people/lena-min.png';">.</a></p>
+<p>I’m a <a href="https://research.fb.com/blog/2020/01/announcing-the-recipients-of-the-2020-facebook-fellowship-awards/"><strong>Facebook PhD Fellow</strong></a> (from fall 2020).</p>
 
-<p>I <a href="https://github.com/yandexdataschool/nlp_course">teach NLP</a> at the <a href="https://yandexdataschool.com">Yandex School of Data Analysis</a>.</p>
+<p>Previously, I was a <strong>research scientist</strong> at <a href="https://research.yandex.com">Yandex Research</a> and worked closely with the <a href="https://translate.yandex.com">Yandex Translate</a> team<a onmouseover="document.fox.src='/img/people/foxie.jpeg';" onmouseout="document.fox.src='/img/people/lena-min.png';">.</a>
+I (still) <a href="https://github.com/yandexdataschool/nlp_course">teach NLP</a> at the <a href="https://yandexdataschool.com">Yandex School of Data Analysis</a>.</p>
 
 <h2 id="news-"><span style="color:darkblue">News </span></h2>
 
 <ul>
+  <li>01/2020 I’m <a href="https://research.fb.com/blog/2020/01/announcing-the-recipients-of-the-2020-facebook-fellowship-awards/">awarded Facebook PhD Fellowhip</a>.</li>
+  <li>01/2020 Giving several talks, including talks at Google Research Berlin, <a href="https://europe.naverlabs.com/research/seminars/analyzing-information-flow-in-transformers/">Naver Labs Europe</a>, NLP track at <a href="https://appliedmldays.org/tracks/ai-nlp">Applied Machine Learning Days at EPFL</a>.</li>
   <li>09/2019 <em>To appear at <strong>NeurIPS</strong> 2019</em>: <a href="https://arxiv.org/abs/1911.00176">Sequence Modeling with Unconstrained Generation Order</a>.</li>
   <li>08/2019 <strong>2</strong> papers <em>accepted to <strong>EMNLP</strong> 2019</em>!</li>
   <li>06/2019 Attending Google NLP summit in Zurich.</li>
   <li>05/2019 <strong>2</strong> papers <em>accepted to <strong>ACL</strong> 2019</em>, one is oral!</li>
-  <li>03/2019 We are happy to host <a href="http://wilkeraziz.github.io">Wilker Aziz</a> and Mirella Lapata at <a href="https://yandex.com/company/blog/nlp-week-2019">Yandex NLP week</a>! More information is <a href="https://academy.yandex.ru/events/data_analysis/NLP_week/">here</a>, video and materials are <a href="https://yandexdataschool.ru/edu-process/nlp-week">here</a></li>
-  <li>09/2018 We are launching a new <a href="https://github.com/yandexdataschool/nlp_course">NLP course</a> at the Yandex School of Data Analysis!</li>
-  <li>04/2018 Our paper <a href="https://arxiv.org/pdf/1805.10163.pdf">Context-Aware Neural Machine Translation Learns Anaphora Resolution</a> has been <em>accepted to ACL 2018</em> and selected for an <em>oral</em> presentation.</li>
 </ul>
 
 
@@ -124,34 +126,36 @@ <h2 id="news-"><span style="color:darkblue">News </span></h2>
       <div class="footer-col  footer-col-1">
         <table >
           <tr class="contact-list">
-            <!-- yandex research-->
+            <!-- yandex research
             <td>
               <div class="media">
                 <a class="pull-left thumbnail" href="https://research.yandex.com">
                   <img src="/img/ico/yr-min.png" height=20px alt=""/>
                 </a>
               </div>
             </td>
-            <!-- UvA-->
+            -->
+            <!-- EdnNLP-->
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="https://www.illc.uva.nl/People/show_person.php?Person_id=Voita+E.">
-                  <img src="/img/ico/uva-min.png" height=20px alt=""/>
+                <a class="pull-left thumbnail" href="https://edinburghnlp.inf.ed.ac.uk/index.php/people/">
+                  <img src="/img/ico/edn_uni-min.png" height=30px alt=""/>
                 </a>
               </div>
             </td>
-            <!-- EdnNLP-->
+            <!-- UvA-->
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="https://edinburghnlp.inf.ed.ac.uk/index.php/people/">
-                  <img src="/img/ico/edn.png" height=20px alt=""/>
+                <a class="pull-left thumbnail" href="https://msclogic.illc.uva.nl/people/students/">
+                  <img src="/img/ico/uva-min.png" height=20px alt=""/>
                 </a>
               </div>
             </td>
 
+
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="mailto:lena-voita@yandex-team.ru">
+                <a class="pull-left thumbnail" href="mailto:lena-voita@hotmail.com">
                   <img src="/img/ico/mail-min.png" height=20px alt=""/>
                 </a>
               </div>
@@ -186,7 +190,7 @@ <h2 id="news-"><span style="color:darkblue">News </span></h2>
       </div> -->
 
       <div class="footer-col  footer-col-3">
-          <p class="text" align="right">Last updated November 26, 2019.</p>
+          <p class="text" align="right">Last updated March 30, 2020.</p>
       </div>
     </div>
 

_site/papers.html

@@ -23,7 +23,7 @@
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
 
     <title>Publications</title>
-    <meta name="description" content="Here you will find details about my work.
+    <meta name="description" content="Hi! I'm Lena and here you will find what I do.
 ">
 
     <link rel="stylesheet" href="/css/main.css">
@@ -68,6 +68,8 @@
 
 
 
+
+
           <a class="page-link" href="/posts.html">Blog</a>
 
 
@@ -102,6 +104,33 @@
       <div class="row">
         <div class="wrapper">
           
+<div class="media">
+    <a class="pull-right">
+    <img src="/img/paper/no_conf.png" alt="" style="max-width:50px; height:auto;" />
+  </a>
+  <a class="pull-left">
+    <img src="/img/paper/mdl_probes-min.png" alt="" style="max-width:120px; height:auto;" />
+  </a>
+  <div class="media-body">
+    <!-- citation -->
+
+    <a href="https://arxiv.org/abs/2003.12298">Information-Theoretic Probing with Minimum Description Length</a><br />
+    <u>Elena Voita</u> and Ivan Titov.
+
+    <br />
+
+
+    <!-- resources -->
+    <!-- draft/slides/poster/talk -->
+
+
+
+    <!-- amendments -->
+
+
+  </div>
+</div>
+
 <div class="media">
     <a class="pull-right">
     <img src="/img/paper/no_conf.png" alt="" style="max-width:50px; height:auto;" />
@@ -324,7 +353,7 @@
     </div>
   <footer class="text-center text-muted">
     <hr/>
-    Last updated November 26, 2019.<br/>
+    Last updated March 30, 2020.<br/>
     <!--Created with
     Based on the code of <a href="https://github.com/alopez/alopez.github.com">Adam Lopez</a>.
     <a href="http://git-scm.com/">git</a>,

_site/posts.html

@@ -7,7 +7,7 @@
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
 
     <title>Blog</title>
-    <meta name="description" content="Here you will find details about my work.
+    <meta name="description" content="Hi! I'm Lena and here you will find what I do.
 ">
 
     <link rel="stylesheet" href="/css/main.css">
@@ -60,6 +60,8 @@
 
 
 
+
+
           <a class="page-link" href="/posts.html">Blog</a>
 
 
@@ -90,6 +92,53 @@
       <div class="wrapper">
         <hr>
 
+<h1 style="font-size:28px;">Information-Theoretic Probing with MDL</h1>
+
+
+<a class="float-right">
+    <img src="../resources/posts/mdl_probes/probe_main_orange-min.png" alt=""
+         style="max-width:350px; height:auto; float: right; margin-left:15px; margin-top:25px"/>
+</a>
+
+
+<span style="font-size:14px;">
+This is a post for the paper
+    <a href="">
+            Information-Theoretic Probing with Minimum Description Length.
+    </a>
+</span>
+
+<br/>
+<br/>
+<span style="font-size:15px;">
+
+    Probing classifiers often fail to adequately reflect  differences in representations
+    and can show different results depending on hyperparameters.
+    </br>
+    As an alternative to the standard probes,
+
+<ul>
+    <li>we propose information-theoretic probing which measures
+        <font face="symbol">minimum description length</font> (MDL) of labels given representations;</li>
+    <li>we show that MDL characterizes both <font face="symbol">probe quality</font> and
+      <font face="symbol">the amount of effort</font> needed to achieve it;</li>
+    <li>we explain how to easily measure MDL on top of standard probe-training pipelines;</li>
+    <li>we show that results of MDL probes are more informative and stable than those of standard probes.</li>
+</ul>
+</span>
+
+<a class="pull-right" href="/posts/mdl_probes.html" onMouseOver="document.readmore4.src='../resources/posts/buttons/button_read_more_push-min.png';" onMouseOut="document.readmore4.src='../resources/posts/buttons/button_read_more-min.png';">
+<img src="../resources/posts/buttons/button_read_more-min.png" name="readmore4" width=120px class="pull-right"></a>
+<a class="pull-right" href="https://arxiv.org/pdf/2003.12298.pdf" onMouseOver="document.readpaper4.src='../resources/posts/buttons/button_read_paper_push-min.png';" onMouseOut="document.readpaper4.src='../resources/posts/buttons/button_read_paper-min.png';">
+<img src="../resources/posts/buttons/button_read_paper-min.png" name="readpaper4" width=120px class="pull-right"></a>
+<a class="pull-right" href="https://github.com/lena-voita/description-length-probing" onMouseOver="document.viewcode4.src='../resources/posts/buttons/button_view_code_push-min.png';" onMouseOut="document.viewcode4.src='../resources/posts/buttons/button_view_code-min.png';">
+<img src="../resources/posts/buttons/button_view_code-min.png" name="viewcode4" width=120px></a>
+
+<span style="font-size:15px; text-align: right; float: right; color:gray">March 2020</span>
+
+<p></p>
+<hr>
+
 
 <!-- <span style="color:darkblue;font-size:28px;">The Story of Heads </span> -->
 <h1 style="font-size:28px;">Evolution of Representations in the Transformer</h1>
@@ -227,34 +276,36 @@ <h1 style="font-size:28px;">The Story of Heads</h1>
       <div class="footer-col  footer-col-1">
         <table >
           <tr class="contact-list">
-            <!-- yandex research-->
+            <!-- yandex research
             <td>
               <div class="media">
                 <a class="pull-left thumbnail" href="https://research.yandex.com">
                   <img src="/img/ico/yr-min.png" height=20px alt=""/>
                 </a>
               </div>
             </td>
-            <!-- UvA-->
+            -->
+            <!-- EdnNLP-->
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="https://www.illc.uva.nl/People/show_person.php?Person_id=Voita+E.">
-                  <img src="/img/ico/uva-min.png" height=20px alt=""/>
+                <a class="pull-left thumbnail" href="https://edinburghnlp.inf.ed.ac.uk/index.php/people/">
+                  <img src="/img/ico/edn_uni-min.png" height=30px alt=""/>
                 </a>
               </div>
             </td>
-            <!-- EdnNLP-->
+            <!-- UvA-->
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="https://edinburghnlp.inf.ed.ac.uk/index.php/people/">
-                  <img src="/img/ico/edn.png" height=20px alt=""/>
+                <a class="pull-left thumbnail" href="https://msclogic.illc.uva.nl/people/students/">
+                  <img src="/img/ico/uva-min.png" height=20px alt=""/>
                 </a>
               </div>
             </td>
 
+
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="mailto:lena-voita@yandex-team.ru">
+                <a class="pull-left thumbnail" href="mailto:lena-voita@hotmail.com">
                   <img src="/img/ico/mail-min.png" height=20px alt=""/>
                 </a>
               </div>
@@ -289,7 +340,7 @@ <h1 style="font-size:28px;">The Story of Heads</h1>
       </div> -->
 
       <div class="footer-col  footer-col-3">
-          <p class="text" align="right">Last updated November 26, 2019.</p>
+          <p class="text" align="right">Last updated March 30, 2020.</p>
       </div>
     </div>
 

_site/posts/acl19_context.html

@@ -7,7 +7,7 @@
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
 
     <title>When a Good Translation is Wrong in Context</title>
-    <meta name="description" content="Here you will find details about my work.
+    <meta name="description" content="Hi! I'm Lena and here you will find what I do.
 ">
 
     <link rel="stylesheet" href="/css/main.css">
@@ -60,6 +60,8 @@
 
 
 
+
+
           <a class="page-link" href="/posts.html">Blog</a>
 
 
@@ -594,34 +596,36 @@ <h2>Conclusions</h2>
       <div class="footer-col  footer-col-1">
         <table >
           <tr class="contact-list">
-            <!-- yandex research-->
+            <!-- yandex research
             <td>
               <div class="media">
                 <a class="pull-left thumbnail" href="https://research.yandex.com">
                   <img src="/img/ico/yr-min.png" height=20px alt=""/>
                 </a>
               </div>
             </td>
-            <!-- UvA-->
+            -->
+            <!-- EdnNLP-->
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="https://www.illc.uva.nl/People/show_person.php?Person_id=Voita+E.">
-                  <img src="/img/ico/uva-min.png" height=20px alt=""/>
+                <a class="pull-left thumbnail" href="https://edinburghnlp.inf.ed.ac.uk/index.php/people/">
+                  <img src="/img/ico/edn_uni-min.png" height=30px alt=""/>
                 </a>
               </div>
             </td>
-            <!-- EdnNLP-->
+            <!-- UvA-->
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="https://edinburghnlp.inf.ed.ac.uk/index.php/people/">
-                  <img src="/img/ico/edn.png" height=20px alt=""/>
+                <a class="pull-left thumbnail" href="https://msclogic.illc.uva.nl/people/students/">
+                  <img src="/img/ico/uva-min.png" height=20px alt=""/>
                 </a>
               </div>
             </td>
 
+
             <td>
               <div class="media">
-                <a class="pull-left thumbnail" href="mailto:lena-voita@yandex-team.ru">
+                <a class="pull-left thumbnail" href="mailto:lena-voita@hotmail.com">
                   <img src="/img/ico/mail-min.png" height=20px alt=""/>
                 </a>
               </div>
@@ -656,7 +660,7 @@ <h2>Conclusions</h2>
       </div> -->
 
       <div class="footer-col  footer-col-3">
-          <p class="text" align="right">Last updated November 26, 2019.</p>
+          <p class="text" align="right">Last updated March 30, 2020.</p>
       </div>
     </div>