Tuesday, May 6, 2025

Optimizing SpringBoot App Deployment with Layers

Stop copying around your dependencies over and over again!!  Most of the time only the application classes change.  Rebuilds should only push the changes! 

Sample docker file for layered extraction of java libraries.  What this does:

  1. extracts your fat spring-boot-generated JAR into separate layer directories
  2. from here, creates a docker image layer by layer
  3. typically only the application/ docker layer will have changed - the rest will load from docker cache

FROM openjdk:17.0.1-jdk-slim as builder

WORKDIR application
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=tools -jar application.jar extract --layers --launcher

FROM openjdk:17.0.1-jdk-slim
WORKDIR application
COPY --from=builder /application/application/dependencies/ ./
COPY --from=builder /application/application/spring-boot-loader/ ./
COPY --from=builder /application/application/snapshot-dependencies/ ./
COPY --from=builder /application/application/application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]


Saturday, March 15, 2025

Working with JHipster Apps

Work in Progress

These have become super sophisticated.

Here are some tips to get started.



Add support for saving a list of objects:

@PostMapping("/products")

public List<ProductDTO> add(@Valid @RequestBody List<ProductDTO> productDTOs) throws URISyntaxException {

        List<ProductDTO> result = productService.saveAll(productDTOs);

        return result;

}

public List<ProductDTO> saveAll(List<ProductDTO> products) {
    return productRepository.saveAll(products); 

}

 add


Wednesday, February 5, 2025

Kubectl Cheat Sheet

Using kubectl within minikube.

----------------------------------------------------------------------
Minikube Installation
----------------------------------------------------------------------

# from the minikube project page
curl -LO https://github.com/kubernetes/minikube/releases/latest/download/minikube-linux-amd64

sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64

ll /usr/local/bin/minikube 

alias kubectl='minikube kubectl --' 

----------------------------------------------------------------------

Cluster Inspection
----------------------------------------------------------------------

kubectl get pods
kubectl get pods -A
kubectl get services 
kubectl get services hello-minikube

kubectl describe pod headlamp-57fb76fcdb-f6bsl
kubectl describe pod headlamp-57fb76fcdb-f6bsl --namespace headlamp


# Show deployed container digests for all pods

kubectl get pods --namespace=mynmspc -o=jsonpath='{range .items[*]}{"\n"}{.metadata.namespace}{","}{.metadata.name}{","}{range .status.containerStatuses[*]}{.image}{", "}{.imageID}{", "}{end}{end}' | sort

----------------------------------------------------------------------
Minikube Control
----------------------------------------------------------------------

minikube [start|stop|pause|unpause]

----------------------------------------------------------------------

Add-Ons and Services
----------------------------------------------------------------------

minikube addons list
# install
minikube addons enable headlamp

# run addon with name
minikube service headlamp -n headlamp

# run addon
minikube service hello-minikube

# open browser to endpoint, when available from addon
minikube addons open <name>

# run addon when mk starts
minikube start --addons <name1> --addons <name2>

----------------------------------------------------------------------
Deploy Service
----------------------------------------------------------------------

kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0

kubectl expose deployment hello-minikube --type=NodePort --port=8080

kubectl get services hello-minikube

kubectl port-forward service/hello-minikube 7080:8080

----------------------------------------------------------------------
Headlamp Setup
----------------------------------------------------------------------

# install addon - see above
minikube addons enable headlamp

# run addon with name
minikube service headlamp -n headlamp

# get login token
kubectl create token headlamp-admin -n kube-system

# metrics-server helps w/admin
#  uses image registry.k8s.io/metrics-server/metrics-server:v0.7.2
minikube addons enable metrics-server

# See for setup instructions
https://headlamp.dev/docs/latest/installation


~                                                    



Saturday, February 1, 2025

New User Setup Script

Adding a new linux user.

----------------------------------------------------------------------
Basic Ubuntu Script
----------------------------------------------------------------------

#!/bin/bash

USERNAME=$1

# Add user and groups
echo "adding user account: $USERNAME"
sudo adduser $USERNAME
sudo usermod -a -G mygroup $USERNAME

# Add sudo (centOS may require wheel group instead)
sudo usermod -a -G sudo $USERNAME

# Download user PKI cert from remote repo - uses server cert
sudo mkdir -p /home/$USERNAME/.ssh
sudo wget -T 90 -t 1 --certificate=/local/server/cert.pem -0 /home/$USERNAME/.ssh/authorized_keys --no-check-certificate https://keyserver/raw/ds/$USERNAME

# Add starter env
sudo cp -p /home/sample/.bashrc /home/$USERNAME/.

# Set perms
sudo chmod -R 700 /home/$USERNAME
sudo chown -R $USERNAME:$USERNAME /home/$USERNAME




Wednesday, August 28, 2024

Python Packaging

Brief History 

Python builds were traditionally done using:
  • distutils - Python built-in, uses requirements.txt to list dependencies, setup.py for configuration
    • ISSUES: split of deps and config could cause conflicts, and load order confusion

  • setuptools - extended distutils, added automatic install of dependencies
Other packaging tools:
  • wheel - PEP 427 - replaced eggs - build tool with better support for C-library integrations
    • creates .pyc files during installation, to match local interpreter
    • supported in ip >= 1.4 and setuptools >= 0.8
  • PyPI - Python Package Index - online registry of built py libs
among others.

Poetry

Adds main project descritor pyproject.toml to contain project metadata, config, and dependencies.

Some features:
  • can declare different dependency sets, i.e. for prod, dev, and test
  • creates file poetry.lock to define frozen dependencies


Useful Commands

> Create a New Project
poetry new my-package

> Installing existing Project - exclude some dependencies
poetry install --without test,docs

> Installing existing Project - only include some dependencies
poetry install --only dev

> Install or Fix existing Project - sync local to canonical
poetry install --sync

> Get Info about an Installed Lib
poetry show <package-name>


TOML Example


From https://towardsdatascience.com/ditch-requirements-use-poetry-00a936fe9b6d:

[tool.poetry]
name = "my_project"
version = "0.1.0"
description = "Description of my project"
package-mode = true
authors = [
"My Team <team@my_company.com>",
]

[tool.poetry.dependencies]
python = "~3.10"
fastapi = "^0.111.1"
pydantic = "^2.8.2"
Jinja2 = "^3.1.2"
uvicorn = {extras = ["standard"], version = "^0.30.3"}

[tool.poetry.group.dev.dependencies]
pytest = "^8.2.2"
black = "^24.3.0"
pre-commit = "^3.8.0"
mypy = "^1.11"
flake8 = "^7.1.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.flake8]
max-line-length = 99
extend-ignore = "E203"

Include From Source

Add the following to include a project in development as a dependency.

[[tool.poetry.source]]
name = "my-artifact-registry-repo"
url = "https://europe-west2-python.pkg.dev/my-gcpproject/my-af-repo/simple"
priority = "explicit"




Wednesday, February 15, 2023

Docker Cheat Sheet


----------------------------------------------------------------------
References
----------------------------------------------------------------------

Docker Engine -- runtime and ops https://docs.docker.com/engine/
Create Docker Containers -- build your own https://docs.docker.com/build/
Orchestrate Containers -- combine containers https://docs.docker.com/compose/

----------------------------------------------------------------------
Basic Docker - Overview
----------------------------------------------------------------------


Basic Docker Architecture

  • Image as container - Binary images of s/w systems published as Containers
  • Versioned - Containers are versioned
  • Decentralized - Your server downloads/stores containers and their parents...warning, this gets large!

Basic Docker Lifecycle

  • Pull containers to use or extend
  • Run containers and expose local resources to them
  • Extend existing containers to create custom run-anywhere binaries

----------------------------------------------------------------------
Basic Docker - Setup
----------------------------------------------------------------------


Install
sudo apt install docker.io
sudo apt install docker-compose


Running without sudo
Assign your user into the docker group. Add group if necessary.

sudo groupadd docker
sudo usermod -a -G docker myuser
# refresh groups or restart server


Other post-install tasks
See https://docs.docker.com/install/linux/linux-postinstall/

Configure - start on boot
On CentOS:
sudo systemctl [enable|disable] docker

----------------------------------------------------------------------
Docker service
----------------------------------------------------------------------

Starting the daemon
sudo service docker start

Start/Stop Docker Service
Debian:
sudo service docker start
CentOS:
sudo systemctl [start|stop] docker 

Test Docker Service
sudo docker run hello-world

----------------------------------------------------------------------
Managing Images
----------------------------------------------------------------------


List all Images
docker images

List all Images for one Artifact
docker images <artifact>

List all Images w/ hashes for one Artifact
docker images --digests <artifact>

Image Info - i.e. How do I use this image?
docker image inspect <imageid>

Rename Image...by aliasing the image
docker tag <imageid> <artifact>:<new tag>

Remove Image (untagging)
docker rmi <alias>

Get Create Date of Image
docker inspect -f '{{ .Created }}' <artifact>:<tag>

Remove Unused Images
docker image prune [-a]

Move docker image location off to another drive (for large files)

sudo service docker stop
ps faux|grep docker
sudo ls /var/lib/docker
mv /var/lib/docker <new location> 
sudo service docker start


----------------------------------------------------------------------
Running Containers
----------------------------------------------------------------------


See running containers
docker ps

Launch container from image
docker run -it [-d] [--rm] [--network <name>]  <image_id> 
where:
-d starts detached (daemon)
--rm removes on exit
--network network name to use (see `docker network ls`)

Retain image changes
docker commit <container_id> <group>/<alias>

Attach and execute shell
docker exec -it <container_id> bash

Build image from local Dockerfile
docker build . -t <alias>

Attach to running shell
docker attach <container_id>

Drop out of running shell - but leave running
ctl-p ctl-d

View container logging
docker logs <container_id>

----------------------------------------------------------------------
Creating Images
----------------------------------------------------------------------


Create a Dockerfile with a base image
Search DockerHub for a suitable base
FROM python:latest

Add some standard labels
LABEL org.opencontainers.image.title="Data Service" \
      org.opencontainers.image.version="0.0.1" \
      org.opencontainers.image.authors="aaa@baz.com,bbb@baz.com"

Add some content
ADD data-svc/ /home/python/data-svc/.

Open port for web service
EXPOSE 8080

Attach volumes for in/out of data
VOLUME ["/home/python/data", "/var/log/data-svc"]

Launch a service on container start
ENTRYPOINT ["flask", "--app", "data-svc", "run"]

Build the Docker Image
From the directory containing Dockerfile...
docker build -t <group>/<artifact>:<version> .

----------------------------------------------------------------------
Docker and Docker-Compose Runtime
----------------------------------------------------------------------

Start docker-compose containers
docker-compose up -d



















Friday, October 7, 2022

My Default vimrc File

 Finally putting this down, because I keep re-creating it.

The .vimrc file which normally goes in your user home dir.  There is a default in the vim install dir as well.

Suggestions welcome!




"For gVim - real GUI
set nocompatible
source $VIMRUNTIME/vimrc_example.vim
source $VIMRUNTIME/mswin.vim
behave mswin

" Disable viminfo
set viminfo=""

" to see all colors:
" ls -al /usr/share/vim/vim7*/colors
":colo evening
:colo blue
":colo slate
":colo darkblue
":colo shine

"set lines=45 columns=120

set tabstop=4 softtabstop=4 shiftwidth=4 expandtab autoindent smarttab
set nocindent
set encoding=utf-8 fileencoding=utf-8
set nobackup nowritebackup noswapfile autoread
set number ruler
set hlsearch incsearch ignorecase smartcase

"bash like auto-completion
set wildmenu
set wildmode=list:longest
"
inoremap
"
" for lusty explorer
noremap glr \lr
noremap glf \lf
noremap glb \lb

" use cntrl-h/j/k/l to switch between splits
map j
map k
map l
map h

" display the status line with the filename
set laststatus=2
set noerrorbells
set foldmethod=syntax
set ls=2
set mouse=v

" disable folding
set nofoldenable