Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Made an mlp baseline that actually scores surprisingly well in ndcg on the large test set. It's competitive with our other top models. #185

Open
wants to merge 9 commits into
base: develop
Choose a base branch
from

Conversation

erikhuck
Copy link

@erikhuck erikhuck commented Sep 5, 2019

closes #184

@erikhuck erikhuck added the enhancement New feature or request label Sep 5, 2019
@erikhuck erikhuck self-assigned this Sep 5, 2019
@codecov-io
Copy link

codecov-io commented Sep 5, 2019

Codecov Report

Merging #185 into develop will increase coverage by 2.1%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           develop     #185     +/-   ##
==========================================
+ Coverage    54.28%   56.38%   +2.1%     
==========================================
  Files           35       35             
  Lines         2487     2497     +10     
==========================================
+ Hits          1350     1408     +58     
+ Misses        1137     1089     -48
Impacted Files Coverage Δ
dna/models/__init__.py 100% <ø> (ø) ⬆️
test/test_models.py 94% <100%> (+0.38%) ⬆️
dna/models/baselines.py 35.71% <100%> (+2.57%) ⬆️
dna/models/base_models.py 63.66% <0%> (+17.26%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2abe98...c28be61. Read the comment docs.

@erikhuck erikhuck assigned bjschoenfeld and unassigned erikhuck Sep 6, 2019
@bjschoenfeld
Copy link
Member

You should not use the final test set until we have had a chance to tune and finalize the model.

Please add some tests to show that the new model is working.

@bjschoenfeld bjschoenfeld assigned erikhuck and unassigned bjschoenfeld Sep 6, 2019
@erikhuck
Copy link
Author

erikhuck commented Sep 6, 2019

@bjschoenfeld I didn't use the final test set. I was referring to the development test set. The one that gets scored every time we run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change MLP to MLPAblation and create new MLP
3 participants