Hi, hope you are doing well.
Recently, I had deployed the ml model on AWS for product.
- Creating EKS cluster using Terraform
- Build docker images for application components like the Frontend, Backend, Model Serving, Model Inference and Database.
- Register docker images to ECR
- Create auto-scaling group for each components, so each one could be auto scaled by the cpu utilization.
- Using Spot instance for Model Serving and Model inference
- Deploy the Cluster AutoScaler using manifest
- Deploy the metric server to get the resource metric
- Deploy Prometheus and Grafana for cluster resource monitoring using Terraform
- Deploy EFK (ElasticSearch, FluentBit and Kibana) to Kubernetes cluster for application debug using Helm
- Building Jenkins pipeline for CI and using ArgoCD for kubernetes deployment
It would be happy if you think these experiences could be helpful for your case.
Looking forward to hear from you.
Thank you.