Skip to content

DataLayer to provide data from LIBSVM format file#1571

Closed
tnarihi wants to merge 11 commits intoBVLC:masterfrom
tnarihi:libsvmdatalayer
Closed

DataLayer to provide data from LIBSVM format file#1571
tnarihi wants to merge 11 commits intoBVLC:masterfrom
tnarihi:libsvmdatalayer

Conversation

@tnarihi
Copy link
Contributor

@tnarihi tnarihi commented Dec 13, 2014

I've just implemented a new layer which provides data from LIBSVM format file.

<label> <index1>:<feature1> ... <index M>:<feature M>

Here is an example of using this layer: http://nbviewer.ipython.org/github/tnarihi/caffe/blob/libsvmdatalayer/examples/libsvm_format_data_into_caffe.ipynb

@sguada
Copy link
Contributor

sguada commented Dec 13, 2014

Adding more data layers is not a sustainable approach. Please take a look at #1568 and try to simplify your approach.

@shelhamer
Copy link
Member

@tnarihi thanks for adding another data format, but as @sguada mentioned we are trying to come up with a more general, less redundant data pipeline. At the same time we are working on making it easier to have "personal" layers that will build with Caffe but don't need to be merged in the main project.

Your libsvm layer could be one of these contributed layers.

@tnarihi
Copy link
Contributor Author

tnarihi commented Dec 17, 2014

@sguada @shelhamer Thanks for the comments ! More general data layer sounds good. I will take a look at the PRs. The "personal" layer concept is also interesting. Is there any PR regarding it?

@shelhamer
Copy link
Member

@tnarihi this could be done by a Python data layer #1703 or through de-centralized / personal layers as discussed in #1896 with a proof-of-concept in #2036. Closing to avoid adding more data layers since the data pipeline needs further work and special formats can be handled by Python. Thanks for sharing your code!

@shelhamer shelhamer closed this Mar 10, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments