Serving up a Real-Time Language Model¶
This example application implements a simple language model of N-grams which are continuously extracted and accumulated from published source documents, all using the Gazette consumer framework.
The complete implementation can be found in package word-count.
Objectives¶
Our goal is to deploy an application which presents a real-time language model service. The model we’ll use is very basic, as a simple collection of bi-gram counts.
It should offer a gRPC API for publishing new documents to be incorporated into the language model, and an API for fetching the current counts of a given bi-gram or prefix. Clients interact with it over its gRPC API – the fact that it’s built as a Gazette consumer could be considered an implementation detail.
Finally, we’ll include a command-line tool wordcountctl
for interacting with the service.
Note
This example has a dependency on RocksDB, which means it cannot be directly
go install
’d without having appropriate development libraries available.
Use the gazette/examples
docker image to run pre-built binaries.
Deploy the Example to Kubernetes¶
Deploy the example to the word-count
namespace of a test Kubernetes cluster,
such as Minikube or Docker for Desktop:
$ kubectl apply -k ./kustomize/test/deploy-word-count/
namespace/word-count created
serviceaccount/minio created
configmap/etcd-scripts-24t872gm88 created
configmap/example-journal-specs created
configmap/gazette-zonemap created
configmap/minio-create-bucket-mm6f469cbt created
configmap/postgres-init created
configmap/publish-docs-word-count-gkhd5f8hdf created
configmap/shard-specs-word-count-5dgc7bg848 created
secret/gazette-aws-credentials created
secret/minio-credentials-fgdm8fkm5m created
service/consumer-word-count created
service/etcd created
service/gazette created
service/minio created
deployment.apps/consumer-word-count created
deployment.apps/gazette created
deployment.apps/minio created
statefulset.apps/etcd created
job.batch/apply-journal-specs created
job.batch/apply-shard-specs-word-count created
job.batch/minio-create-bucket created
job.batch/publish-docs-word-count created
The Kustomize manifest starts the consumer-word-count
deployment, its
dependencies, jobs to create specifications, and a job publish-docs-word-count
which
publishes a small set of documents to the application.
Query for Counts¶
Port-forward to the consumer in a new tab:
$ kubectl -nword-count port-forward svc/consumer-word-count 32190:8080
Then use docker to run wordcountctl and query for a specific bi-gram. The consumer process will proxy the request to the appropriate shard:
$ docker run --network host --env CONSUMER_ADDRESS=http://localhost:32190 \
--rm -it gazette/examples wordcountctl query --prefix="the city"
We can also query a specific shard for all of its bi-grams having a prefix:
$ docker run --network host --env CONSUMER_ADDRESS=http://localhost:32190 \
--rm -it gazette/examples wordcountctl query --prefix="the" --shard=shard-000