GRPC / Protocol Buffer support

[vanjad-sixt]

Hello Tomasz (or anyone else, for that matter),

We are considering using specification argument resolver in a GRPC setting, using protobuf definitions as contracts to RPC handlers.

I would like to attempt an extension to the library, which would convert proto messages into specifications, (similar to how jsonPath currently works).

Could you perhaps give me a rough idea, how one could tackle this?

Many thanks,

Vanja

[jradlica]

Hello @vanjad-sixt ,

I think that the simplest way to use specification-arg-resolver with gRPC/Protobuf is by leveraging the
SpecificationBuilder inside your GrpcService logic.

Example 1:

CustomerGrpcService.java

package net.kaczmarzyk;

import net.kaczmarzyk.spring.data.jpa.Customer;
import net.kaczmarzyk.spring.data.jpa.CustomerRepository;
import net.kaczmarzyk.spring.data.jpa.domain.Equal;
import net.kaczmarzyk.spring.data.jpa.utils.SpecificationBuilder;
import net.kaczmarzyk.spring.data.jpa.web.annotation.Spec;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.jpa.domain.Specification;
import org.springframework.stereotype.Service;

import java.util.List;

import static net.kaczmarzyk.spring.data.jpa.utils.SpecificationBuilder.specification;

@Service
public class CustomerGrpcService extends CustomerServiceGrpc.CustomerServiceImplBase {

	@Spec(path = "id", jsonPaths = "customerId", spec = Equal.class)
	interface CustomerSpec extends Specification<Customer> {

	}

	@Autowired
	private CustomerRepository customerRepository;

	@Override
	public void findById(CustomerSearch.CustomerIdRequest request, io.grpc.stub.StreamObserver<CustomerSearch.CustomerList> responseObserver) {
		SpecificationBuilder<CustomerSpec> spec = specification(CustomerSpec.class);
		spec.withJsonBodyParam("id", request.getCustomerId());
		spec.withJsonBodyParam("name", request.getName());

		List<Customer> customers = customerRepository.findAll(spec.build());

		//TODO protobuf logic
	}
}

Customer.proto

syntax = "proto3";

package net.kaczmarzyk;

option java_package = "net.kaczmarzyk";
option java_outer_classname = "CustomerSearch";

message CustomerIdRequest {
  string customerId = 1;
  string name = 2;
}

message Customer {
  int64 id = 1;
  string first_name = 2;
  string last_name = 3;
  string gender = 4;
  string email = 5;
}

message CustomerList {
  repeated Customer customers = 1;
}

service CustomerService {
  rpc FindById (CustomerIdRequest) returns (CustomerList);
}

Example 2 (more generic approach):

CustomerGrpcService.java

package net.kaczmarzyk;

import net.kaczmarzyk.spring.data.jpa.Customer;
import net.kaczmarzyk.spring.data.jpa.CustomerRepository;
import net.kaczmarzyk.spring.data.jpa.domain.Equal;
import net.kaczmarzyk.spring.data.jpa.utils.CustomerServiceGrpc;
import net.kaczmarzyk.spring.data.jpa.utils.GenericProtobufMessage;
import net.kaczmarzyk.spring.data.jpa.utils.SpecificationBuilder;
import net.kaczmarzyk.spring.data.jpa.web.annotation.Spec;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.jpa.domain.Specification;
import org.springframework.stereotype.Service;

import java.util.List;

import static net.kaczmarzyk.spring.data.jpa.utils.SpecificationBuilder.specification;

@Service
public class CustomerGrpcService extends CustomerGenericSearchServiceGrpc.CustomerGenericSearchServiceImplBase {

	@Spec(path = "id", jsonPaths = "customerId", spec = Equal.class)
	interface CustomerSpec extends Specification<Customer> {

	}

	@Autowired
	private CustomerRepository customerRepository;

	@Override
	public void findById(CustomerGenericSearch.GenericRequest request, io.grpc.stub.StreamObserver<CustomerGenericSearch.CustomerList> responseObserver) {
		SpecificationBuilder<CustomerSpec> spec = specification(CustomerSpec.class);
		request.getFieldsMap().forEach((key, value) -> {
			spec.withArg(key, value);
		});

		List<Customer> customers = customerRepository.findAll(spec.build());

		//TODO protobuf logic
	}
}

Customer.proto

syntax = "proto3";

package net.kaczmarzyk;

option java_package = "net.kaczmarzyk";
option java_outer_classname = "CustomerGenericSearch";

message GenericRequest {
  map<string, string> fields = 1;
}

message Customer {
  int64 id = 1;
  string first_name = 2;
  string last_name = 3;
  string gender = 4;
  string email = 5;
}

message CustomerList {
  repeated Customer customers = 1;
}

service CustomerGenericSearchService {
  rpc FindById (GenericRequest) returns (CustomerList);
}

[vanjad-sixt]

Hi @jradlica , thanks for the quick answer!
That actually looks quite clean 🤩

After fiddling a bit (and perhaps abusing internals), I got to this idea, which goes in the generic direction, and allows sub-paths:

ProtoSpecificationFactory

import com.google.protobuf.Descriptors.FieldDescriptor;
import com.google.protobuf.Message;
import net.kaczmarzyk.spring.data.jpa.utils.QueryContext;
import net.kaczmarzyk.spring.data.jpa.web.DefaultQueryContext;
import net.kaczmarzyk.spring.data.jpa.web.ProcessingContext;
import net.kaczmarzyk.spring.data.jpa.web.SpecificationFactory;
import net.kaczmarzyk.spring.data.jpa.web.annotation.MissingPathVarPolicy;
import org.springframework.data.jpa.domain.Specification;

import java.lang.annotation.Annotation;
import java.util.ArrayList;
import java.util.Locale;

@Component
public class ProtoSpecificationFactory {
    private final SpecificationFactory specificationFactory;

    public ProtoSpecificationFactory() {
        this.specificationFactory = new SpecificationFactory(null, null, Locale.ENGLISH);

    }

    @SuppressWarnings("unchecked")
    public <T extends Specification<?>> T createSpecification(Class<T> specInterface, Message message) {
        return (T) specificationFactory.createSpecificationDependingOn(
                new ProtoProcessingContext(specInterface, message));
    }

    public static class ProtoProcessingContext implements ProcessingContext {

        public static final Annotation[] EMPTY_ANNOTATIONS = {};
        public static final DefaultQueryContext EMPTY_QUERY_CONTEXT = new DefaultQueryContext();
        private final Class<?> parameterType;
        private final Message message;

        public ProtoProcessingContext(Class<?> parameterType, Message message) {
            this.parameterType = parameterType;
            this.message = message;
        }

        @Override
        public String[] getBodyParamValues(String path) {
            Message currentMessage = message;
            ArrayList<String> strings = new ArrayList<>();
            for (String segment : path.split("\\.")) {
                FieldDescriptor fieldDescriptor = currentMessage.getDescriptorForType().findFieldByName(segment);
                if (fieldDescriptor == null) {
                    throw new IllegalStateException("Field " + segment + " not found in message " + currentMessage.getDescriptorForType().getFullName());
                }
                if (fieldDescriptor.getType() == FieldDescriptor.Type.MESSAGE) {
                    currentMessage = (Message) currentMessage.getField(fieldDescriptor);
                    continue;
                }
                if (fieldDescriptor.isRepeated()) {
                    for (Object value : (Iterable<?>) currentMessage.getField(fieldDescriptor)) {
                        strings.add(value.toString());
                    }
                    continue;
                }
                strings.add(currentMessage.getField(fieldDescriptor).toString());
            }
            return strings.toArray(new String[0]);
        }
    }

        @Override
        public Class<?> getParameterType() {
            return this.parameterType;
        }

        @Override
        public Annotation[] getParameterAnnotations() {
            return EMPTY_ANNOTATIONS;
        }

        @Override
        public QueryContext queryContext() {
            return EMPTY_QUERY_CONTEXT;
        }

        @Override
        public String getRequestHeaderValue(String s) {
            throw new IllegalStateException("Protobuf messages do not have headers.");
        }

        @Override
        public String[] getParameterValues(String s) {
            throw new IllegalStateException("Protobuf messages do not have parameters.");
        }

        @Override
        public String getPathVariableValue(String s, MissingPathVarPolicy missingPathVarPolicy) {
            throw new IllegalStateException("Protobuf messages do not have path variables.");
        }
}

With intended usage as:

CustomerGrpcService

import com.sixt.service.vehicle.lib.ProtoSpecificationFactory;
import net.kaczmarzyk.spring.data.jpa.Customer;
import net.kaczmarzyk.spring.data.jpa.CustomerRepository;
import net.kaczmarzyk.spring.data.jpa.domain.Equal;
import net.kaczmarzyk.spring.data.jpa.web.annotation.Spec;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.jpa.domain.Specification;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class CustomerGrpcService {

    @Spec(
            // this is as usual
            path = "address.street",
            // proto message has e.g. "filter" sub-message with "repeated string addressStreet" prop
            jsonPaths = "filter.addressStreet",
            // search for multiple streets
            spec = In.class)
    interface CustomerSpec extends Specification<Customer> { }
    
    @Autowired
    private ProtoSpecificationFactory protoSpecFactory;

    @Autowired
    private CustomerRepository customerRepository;

    @Override
    public void findById(CustomerRequest request) {
        Specification<Customer> spec = protoSpecFactory.createSpecification(CustomerSpec.class, request);

        List<Customer> customers = customerRepository.findAll(spec);

        //TODO protobuf logic
    }
}

customer-service.proto

service CustomerService {
  rpc Find (CustomerRequest) returns (CustomerList);
}

message CustomerRequest {
  Filter filter = 1;
  Page page = 2;
  message Filter {
    repeated string addressStreet = 1;
  }
  message Page {
    int32 size = 1;
    int32 number = 2;
  }
}

message Customer {
  int64 id = 1;
  // ... other fields
}

message CustomerList {
  repeated Customer customers = 1;
}

Am I off base here?

P.S.: Keep in mind it's a quick POC draft.

[jradlica]

In my opinion, it seems to be a good direction :)

If you have time, feel free to create a PR. Ideally, it should:

• include tests
• ensure that support for Protobuf is optional
• update the README