forked from igormolybog/igormolybog.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
267 lines (199 loc) · 14.6 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta name="generator" content="jemdoc, see http://jemdoc.jaboc.net/" />
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<link rel="stylesheet" href="jemdoc.css" type="text/css" />
<title>Igor Molybog</title>
</head>
<body>
<table summary="Table for page layout." id="tlayout">
<tr valign="top">
<td id="layout-menu">
<div class="menu-category">Menu</div>
<div class="menu-item"><a href="index.html" class="current">Home</a></div>
<div class="menu-item"><a href="research.html">Research</a></div>
<div class="menu-item"><a href="CV.pdf">CV</a></div>
<div class="menu-item"><a href="https://scholar.google.com/citations?user=aJUg17UAAAAJ&hl=en">Publications</a></div>
<div class="menu-item"><a href="https://github.com/igormolybog">GitHub</a></div>
<div class="menu-category">Teaching</div>
<div class="menu-item"><a href="ece491B-693B_fall2024.html">ECE491B/693B</a></div>
<div class="menu-item"><a href="ee417-617_spring2024.html">EE417/617</a></div>
<div class="menu-item"><a href="ee650_fall2023.html">EE650</a></div>
<div class="menu-item"><a href="ieor169.html">IEOR169</a></div>
<div class="menu-item"><a href="ieor240.html">IEOR240</a></div>
<div class="menu-item"><a href="https://calendar.app.google/mLp9CvS96utpEjDK6">Book an Appointment</a></div>
</td>
<td id="layout-content">
<div id="toptitle">
<h1>Igor Molybog</h1>
<div id="subtitle">
</div>
</div>
<table class="imgtable"><tr><td>
<img src="myport.jpg" alt="" width="260px" height="260px" /> </td>
<td align="left"><p><u>Igor Molybog</u> <br />
Researcher in the domains of Numerical Optimization and Machine Learning <br />
Assistant Professor at the University of Hawai'i at Manoa <br />
Office: POST 205H <br />
Email: <i>igormolybog AT gmail.com</i> <br />
<br />
<br />
I am looking for exceptional students and postdocs to join <a href="https://docs.google.com/document/d/1-syg8PM6zR0iX03Zyci_kTxFPr1w3BQXLEpxc3jqKSc/edit?usp=sharing" target=“blank”>my research group</a>!<br />
If you are a prospective or admitted graduate student interested in joining my group, please <a href="https://forms.gle/3mRCfpKQMzCskRfc8" target=“blank”>submit the form</a>.
</p>
</td></tr></table>
<h2>About me</h2>
<p>I am a researcher working on the training of Generative Artificial Intelligence. My research
interests cover the area of computational techniques for learning from data
that are highly scalable with the availability of the compute. More specifically, I am working on optimizers
and model architecture scaling schemes that would allow predictable
and efficient training of Transformer models containing hundreds of billions of parameters.
<br />
In 2022, I graduated with a Ph.D. from the group of <a href="http://www.ieor.berkeley.edu/~lavaei/" target=“blank”>Professor Javad Lavaei</a> at the Department of
Industrial Engineering and Operations Research at University of California, Berkeley, where I worked
on problems of algorithmic analysis and optimal control of complex safety-critical systems, such as power systems,
transportation and telecommunication networks, AI recommendation and navigation systems, robotic systems, and others.
My research spanned the theory of non-convex and conic optimization, stochastic control, machine learning,
and computational and sampling complexity of learning algorithms. I designed data processing algorithms
that are robust to noise and highly scalable with the amount of available computational resources.
</p>
<h2>News</h2>
<p>
</p>
<ul>
<li><p>January 2024: I am presenting the overview of my research to the Department of Information and Computer Sciences at UH Manoa on Tuesday, Jan 9, 2024, 12:00-13:00, in Keller Hall 103. (<a href="https://drive.google.com/file/d/1MFz1Et4EcjOx6eGp04DsEbDtZ9hk_BL6/view?usp=sharing" target=“blank”>recording</a>)
</p>
</li>
<li><p>January 2024: We are delighted to announce the <a href="https://haw-aii.github.io" target=“blank”>Hawai'i Artificial Intelligence Initiative</a> aiming to accelerate AI research at UHM by fostering collaboration
</p>
</li>
<li><p>December 2023: On 18 Dec, we are happy to host <a href="https://www.xiao-li.org/" target=“blank”>Prof. Xiao Li from CUHK</a> with a talk on Convergence Guarantees for SGD with Random
Reshuffling. Please join us at HH 386 at 11 am! (<a href="https://drive.google.com/file/d/1-NwZPMuimE3ZwhTpK5IkfZxfAB49EkDx/view?usp=sharing" target=“blank”>recording</a>)
</p>
</li>
<li><p>October 2023: I am chairing session WA67 “Challanges in the Large-scale Model Training” at INFORMS Annual Meeting 2023!
</p>
</li>
<li><p>October 2023: I have presented a talk on the Large Language Modeling practice at the CS Department seminar of UH Manoa: <a href="https://drive.google.com/file/d/19H5-6PspYAjf4sgSefM0HrnjgRgj1060/view?usp=sharing" target=“blank”>recording</a>, <a href="https://drive.google.com/file/d/1X4KBCU0DiHtQ0aJrLlYKe8509hZFilcY/view?usp=sharing" target=“blank”>slides</a>
</p>
</li>
<li><p>September 2023: A new paper on extending the context length of a trained RoPE transformer: <a href="https://arxiv.org/pdf/2309.16039.pdf" target=“blank”>Effective Long-Context Scaling of Foundation Models</a>
</p>
</li>
<li><p>July 2023: Our paper <a href="https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/" target=“blank”>Llama 2: Open Foundation and Fine-Tuned Chat Models</a> landed with a splash, <a href="https://news.ycombinator.com/item?id=36774627" target=“blank”>making it to the top</a> of hackernews.
</p>
</li>
<li><p>May 2023: I will present the paper <a href="https://arxiv.org/abs/2304.09871" target=“blank”>A Theory on Adam Instability in Large-Scale Machine Learning</a> at the FAIR <> GenAI Workshop. <a href="https://docs.google.com/presentation/d/1MAcgmCgx5b6zf9So42Bh-af9HvOGdBkUrJHKmJPp4IU/edit?usp=sharing" target=“blank”>slides</a>
</p>
</li>
<li><p>April 2023: Our paper <a href="https://people.eecs.berkeley.edu/~sojoudi/Saddle_2023_1.pdf" target=“blank”>Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points</a> got accepted at the 2023 International Conference on Machine Learning (ICML).
</p>
</li>
<li><p>March 2023: New paper <a href="https://arxiv.org/abs/2304.09871" target=“blank”>A Theory on Adam Instability in Large-Scale Machine Learning</a> is accessible online.
</p>
</li>
<li><p>January 2023: New paper <a href="https://people.eecs.berkeley.edu/~sojoudi/Saddle_2023_1.pdf" target=“blank”>Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points</a> is accessible online.
</p>
</li>
<li><p>December 2022: I am going to join the <a href="https://www.ee.hawaii.edu/" target=“blank”>Department of Electrical and Computer Engineering at University of Hawai'i at Manoa</a> as a faculty member!
</p>
</li>
<li><p>October 2022: I am invited to organize a session “Large-scale smooth optimization for Generative AI” at the <a href="https://meetings.informs.org/wordpress/phoenix2023/" target=“blank”>INFORMS Annual Meeting 2023</a>.
</p>
</li>
<li><p>June 2022: I am joining <a href="https://ai.facebook.com/" target=“blank”>Meta AI (FAIR)</a> as a Research Scientist specializing on large-scale optimization.
</p>
</li>
<li><p>May 2022: I defended my thesis and graduated with a Ph.D. in Engineering!
</p>
</li>
<li><p>April 2022: My Thesis <a href="https://escholarship.org/content/qt9jx9w0mh/qt9jx9w0mh.pdf" target=“blank”>The complexity of non-convex and conic optimization problems in data science applications</a> is available online.
</p>
</li>
<li><p>February 2022: I will give a talk at the Department of Electrical and Computer Engineeting of University of Hawai'i at Manoa. <a href="https://docs.google.com/presentation/d/1fPviatr3hesi-hL6GkUffwp4uvJizSQSZ2fuSELAus8/edit?usp=sharing" target=“blank”>slides</a>
</p>
</li>
<li><p>December 2021: I will give a talk “Computation-information complexity trade-off in Tensor PCA” Random Matrices and Random Landscapes seminar at Mathematical Sciences Research Institute, Berkeley, CA. (<a href="Rand_Mat___Land_Project_Presentation_Dec_2021.pdf" target=“blank”>slides</a>)
</p>
</li>
<li><p>November 2021: I will give a talk “Topological complexity of polynomials” at the Control and Optimization seminar, IEOR Department at University of California, Berkeley. (<a href="Presentation_Spin_Glass_November_2021.pdf" target=“blank”>slides</a>)
</p>
</li>
<li><p>September 2021: I am organising session “Reaching Global Optimum in Non-Convex Optimization Problems” at the INFORMS Annual Meeting 2021.
</p>
</li>
<li><p>July 2021: I will present the paper <a href="https://arxiv.org/pdf/2006.00453" target=“blank”>When Does MAML Objective Have Benign Landscape?</a> on the 2021 IEEE Conference on Control Technology and Applications (CCTA).
</p>
</li>
<li><p>May 2021: I will present the paper <a href="https://people.eecs.berkeley.edu/~sojoudi/SRIP_21.pdf" target=“blank”>No spurious solutions in non-convex matrix sensing: Structure compensates for isometry</a> on the 2021 American Control Conference (ACC).
</p>
</li>
<li><p>December 2020: Our paper <a href="https://people.eecs.berkeley.edu/~sojoudi/KSP_2019_1.pdf" target=“blank”>Role of sparsity and structure in the optimization landscape of non-convex matrix sensing</a> got accepted for publication in <a href="https://www.springer.com/journal/10107" target=“blank”>Mathematical Programming</a>.
</p>
</li>
<li><p>September 2020: Our paper <a href="https://www.jmlr.org/papers/volume21/18-881/18-881.pdf" target=“blank”>Conic Optimization for Quadratic Regression Under Sparse Noise</a> got accepted for publication in the <a href="https://www.jmlr.org/" target=“blank”>Journal of Machine Learning Research</a>.
</p>
</li>
<li><p>June 2020: Our new paper <a href="https://lavaei.ieor.berkeley.edu/Meta_2020_1.pdf" target=“blank”>Global convergence of MAML for LQR</a> is accessible online.
</p>
</li>
<li><p>August 2019: I gave a talk “Frontiers of Deep Learning: overview of Simon's Institute summer workshops” at the Control and Optimization seminar, IEOR Department at University of California, Berkeley. (<a href="simonsTalk.pdf" target=“blank”>slides</a>)
</p>
</li>
<li><p>July 2019: The paper <a href="http://www.ieor.berkeley.edu/~lavaei/linear-SE_2019_2.pdf" target=“blank”>Towards Robust and Scalable Power System State Estimation</a> to appear in Proc. 58th IEEE Conference on Decision and Control
</p>
</li>
<li><p>May 2019: New paper on non-convex learning: <a href="http://www.ieor.berkeley.edu/~lavaei/SRIP_2019_1.pdf" target=“blank”>No Spurious Solutions in Non-convex Matrix Sensing: Structure Compensates for Isometry</a>
</p>
</li>
<li><p>January 2019: New paper on data analytics: <a href="http://www.ieor.berkeley.edu/~lavaei/Conic_RQR_2018.pdf" target=“blank”>Conic Optimization for Robust Quadratic Regression</a>
</p>
</li>
<li><p>December 2018: Our paper <a href="http://lavaei.ieor.berkeley.edu/Sampling_SDP_2018.pdf" target=“blank”>On Sampling Complexity of the Semidefinite Affine Rank Feasibility Problem</a> has been designated for oral presentation on <a href="https://aaai.org/Conferences/AAAI-19/" target=“blank”>Thirty-Third AAAI Conference on Artificial Intelligence</a>
</p>
</li>
<li><p>November 2018: I will give a talk “Conic Optimization For Robust Quadratic Regression” at the <a href="https://cdc2018.ieeecss.org/" target=“blank”>57th IEEE Conference on Decision and Control</a>
</p>
</li>
<li><p>October 2018: Our paper <a href="http://www.ieor.berkeley.edu/~lavaei/Sampling_SDP_2018.pdf" target=“blank”>On Sampling Complexity of the Semidefinite Affine Rank Feasibility Problem</a> was accepted on <a href="https://aaai.org/Conferences/AAAI-19/" target=“blank”>Thirty-Third AAAI Conference on Artificial Intelligence</a>
</p>
</li>
<li><p>September 2018: I successfully passed Doctoral Qualifying Examination.
</p>
</li>
<li><p>September 2018: I gave a talk “Geometry of SDP relaxations for rank constrained problems” at the Power Systems Seminar for IEOR Department at University of California, Berkeley.
</p>
</li>
<li><p>September 2018: New paper on SDP relaxations of rank-constrained problems: <a href="http://lavaei.ieor.berkeley.edu/Sampling_SDP_2018.pdf" target=“blank”>On Sampling Complexity of the Semidefinite Affine Rank Feasibility Problem</a>.
</p>
</li>
<li><p>July 2018: Our paper <a href="http://www.ieor.berkeley.edu/~lavaei/regression_2018_1.pdf" target=“blank”>Conic Optimization for Robust Quadratic Regression: Deterministic Bounds and Statistical Analysis</a> to appear in IEEE Conference on Decision and Control, 2018.
</p>
</li>
<li><p>July 2018: I will give a talk on “Conic Optimization For Robust State Estimation: Deterministic Bounds And Statistical Analysis” at <a href="http://meetings2.informs.org/wordpress/phoenix2018/" target=“blank”>INFORMS Annual Meeting</a>
</p>
</li>
<li><p>May 2018: I successfully passed Ph.D. Preliminary Exam.
</p>
</li>
<li><p>April 2018: I gave a talk on “Conic Relaxations for State Estimation under Sparse Noise” at the Power Systems Seminar for IEOR Department at University of California, Berkeley.
</p>
</li>
<li><p>March 2018: New paper on robust nonlinear regression for bad data detection: <a href="http://www.ieor.berkeley.edu/~lavaei/regression_2018_1.pdf" target=“blank”>Conic Optimization for Robust Quadratic Regression: Deterministic Bounds and Statistical Analysis</a>.
</p>
</li>
<li><p>August 2017: I joined the department of <a href="http://www.ieor.berkeley.edu" target=“blank”>Industrial Engineering and Operations Research</a> at <a href="http://www.berkeley.edu" target=“blank”>University of California, Berkeley</a> as a M.Sc/PhD Scholar.
</p>
</li>
</ul>
</td>
</tr>
</table>
</body>
</html>