Skip to content

Merge hcq-embedding-3 #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
minafarid merged 49 commits into from
Sep 26, 2019
Merged

Merge hcq-embedding-3 #97

minafarid merged 49 commits into from
Sep 26, 2019

Conversation

@minafarid
Copy link
Contributor

@minafarid minafarid commented Sep 26, 2019

No description provided.

richardwu and others added 30 commits June 22, 2019 01:12
@richardwu
Relocated test data into subdirectories.
02f8a4f
@richardwu
Move active attributes to right after error detection and inside
99c45f6 
Dataset. Move correlations to separate module.
@richardwu
Remove 2.7 tests from Travis because of Python 3 features.
da83103
@richardwu
Refactor domain generation sort domain by co-occurrence probability and
a340cac 
also domain generation for tuple embedding model.
@richardwu
Make co-occurrence featurizer only generate co-occurrence features for
96080c1 
active attributes. Refactored domain to run estimator separately from
domain generation.
@richardwu
Implemented TupleEmbedding model as an estimator.
673a779
@richardwu
Always load clean/ground truth as strings since we load/store raw dat…
78849ed 
…a as

strings.
@richardwu
Added featurizer for learned embeddings from TupleEmbedding model.
44afd2d
@richardwu
Support multiple layers during repair and made TupleEmbedding dump/load
a6e5453 
more sophisticated.
@zaqthss @richardwu
Quantization and handling of numerical/mixed data.
8b2e076
@richardwu
Improved validation logging and fixed a few bugs.
116e656
@richardwu
Improve validation in TupleEmbedding using pandas dataframes.
e2bc74c
@richardwu
Suppose multi-dimensional quantization.
ce2dc33
@richardwu
Quantize from dict rather than numerical attrs.
33be8a4
@richardwu
Mean/var normalize numerical attributes in context and added
67547f5 
non-linearity to numerical spans.
@richardwu
Support specifying n-dimensional numerical attr groups vs splitting on
61b4147 
columns.
@richardwu
Fixed None numerical_attr_groups.
cd0fd9a
@richardwu
Fixed report RMS error and converting to floats for quantization.
677e602
@richardwu
Added store_to_fb flag to load_data, added LR schedule to
22950d4 
TupleEmbedding, added multiple ground truth in evaluation, changed
EmbeddingFeat to return probability instead of embedding vectors.
@richardwu
Pre-split domain and ground truth values.
873a779
@richardwu
Fixed batch size argument in EmbeddingFeaturizer.
2ba3116
@richardwu
Removed numerical_attrs reference from Table.
6bc733e
@richardwu
Fix to how multi-ground truth is handled. Use simplified numerical
d902be8 
regression TupleEmbedding with nonlinearity.
@richardwu
Max domain size need only be as large as largest for categorical
4a43c6a 
attributes.
@richardwu
Remove domain for numerical attributes in TupleEmbedding.
123845d
@richardwu
Fixed some reference issues and added infer all mode.
186099e
@richardwu
Fixed _nan_ replacement, max_cat_domain being possibly nan, and
71c0e37 
evaluation for sample accuracy.
@richardwu
Do not weak label clean cells and fixed raw data in Logistic estimator.
4c5cd30
@richardwu
Added ReLU after context for numerical targets in TupleEmbedding and
bc3a3ed 
refactored EmbeddingFeat to support numerical feature (RMSE) from
TupleEmbedding.
@richardwu
Use cosine annealing with restart LR schedule and use weak_label instead
69ba100 
of init.
richardwu and others added 19 commits June 22, 2019 01:12
@richardwu
Fixed memory issues with get_features and predict_pp_batch.
120ab08
@richardwu
Fixed bug in get_features.
36e4fff
@richardwu
Added comment to EmbeddingFeat.
4dca0e9
@richardwu
Finally fixed memory issues with torch.no_grad.
3792df0
@richardwu
ConstraintFeaturizer runs on un-quantized values.
d22c91a
@richardwu
Do not drop single value cells (for evaluation).
b13ccb2
@richardwu
Do not generate queries/feature for DC that does not pertain to
f9c14f1 
attributes we are training on.
@richardwu
Fixed ConstraintFeaturizer to handle no DCs.
dc7433a
@richardwu
Removed deprecated code and added dropout.
1d9b92c
@richardwu
Fixed calculation of num_batches in learning loop.
c0d7f0b
@zaqthss @richardwu
do not drop null inits cells with dom(len) <= 1
aa26704
@richardwu
Fixed z-scoring with 0 std and deleting e-notation numerical values.
cbd7dca
@richardwu
Do not quantize if bins > unique.
aea74a5
@richardwu
Fixed some things in domain.
05f04bf
@richardwu
Added repair w/ validation set and removed multiple correct values in
6e53187 
evaluation.
@richardwu
Fixed domain generation to include single value cells in domain.
ad0bd4c
@richardwu
Handle untrained context values properly and added code for domain
59b4e3f 
co-occurrence in tupleembedding.
@richardwu
Regression fix for moving raw_data_dict before z-normalization and
ddbf829 
removed code references to domain_cooccur (for the most part).
@minafarid
Merge branch 'dev' into hcq-embedding-3
1e85bbd
@minafarid minafarid merged commit ae30186 into dev Sep 26, 2019
@minafarid minafarid deleted the hcq-embedding-3 branch September 26, 2019 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@minafarid @richardwu @zaqthss