Superbowl AI

Did you catch that the NFL is using AI image processing to help settle in-game calls like field position?

Seems like some welcome backup for referees.

I'm waiting for the leagues to start regulating performance-enhancing AI. Otherwise sports will become a game of AI v. AI,  like  chess, AlphaGo, and now Core War.

Oh - and we'll have to figure out how to test for that.  

AI in Core Wars

Of all the AI stories happening around us, this one seems most telling to me.

An AI mastering the infinite strategic and tactical space of the Core War programming challenge makes the step into autonomous bots attacking and defending network boundaries seem very close.

[need to see where the state of the art is in that space]

If you're an old developer like me, then you may remember the Core War programming challenge.
Basically computer viruses battling for control of a machine.
The game essentially models an infinite strategic and tactical space.

A Japanese research team has shown that an AI can be trained to be exceptionally competitive - including learning in hours the same winning strategies that humans have taken years to craft.

https://pub.sakana.ai/drq/

Buckle down your networks, and deploy your AI hounds!

Optimizing SpringBoot App Deployment with Layers

Stop copying around your dependencies over and over again!!  Most of the time only the application classes change.  Rebuilds should only push the changes! 

Sample docker file for layered extraction of java libraries.  What this does:

1. extracts your fat spring-boot-generated JAR into separate layer directories
2. from here, creates a docker image layer by layer
3. typically only the application/ docker layer will have changed - the rest will load from docker cache

FROM openjdk:17.0.1-jdk-slim as builder

WORKDIR application
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=tools -jar application.jar extract --layers --launcher

FROM openjdk:17.0.1-jdk-slim
WORKDIR application
COPY --from=builder /application/application/dependencies/ ./
COPY --from=builder /application/application/spring-boot-loader/ ./
COPY --from=builder /application/application/snapshot-dependencies/ ./
COPY --from=builder /application/application/application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]

Working with JHipster Apps

Work in Progress

These have become super sophisticated.

Here are some tips to get started.

Add support for saving a list of objects:

@PostMapping("/products")

public List<ProductDTO> add(@Valid @RequestBody List<ProductDTO> productDTOs) throws URISyntaxException {

        List<ProductDTO> result = productService.saveAll(productDTOs);

        return result;

}

public List<ProductDTO> saveAll(List<ProductDTO> products) {
    return productRepository.saveAll(products); 

}

 add

Kubectl Cheat Sheet

Canonical guide:  https://kubernetes.io/docs/reference/kubectl/quick-reference/

----------------------------------------------------------------------
Cluster Inspection
----------------------------------------------------------------------

kubectl get pods
kubectl get pods -A
kubectl get services 
kubectl get services hello-minikube

kubectl describe pod headlamp-57fb76fcdb-f6bsl
kubectl describe pod headlamp-57fb76fcdb-f6bsl --namespace headlamp

# Show deployed container digests for all pods

kubectl get pods --namespace=mynmspc -o=jsonpath='{range .items[*]}{"\n"}{.metadata.namespace}{","}{.metadata.name}{","}{range .status.containerStatuses[*]}{.image}{", "}{.imageID}{", "}{end}{end}' | sort

# Locate pods for a service/deployment

kubectl get pods -n <namespace> |grep <name>

# Locate pods for a service

kubectl describe service <service name> -n <namespace>

<identify label>

kubectl get pods -l app.kubernetes.io/<label>=<value>

# Follow log for a pod

kubectl logs -f <podname> -n <namespace>

# Follow logs for multiple pods

kubectl logs -l app.kubernetes.io/<label>=<value> -n <namspace> 

> e.g. follow all logs under the instance selector

kubectl logs -l app.kubernetes.io/instance=rke2-ingress-nginx -n kube-system 

----------------------------------------------------------------------
Download Deployment descriptors
----------------------------------------------------------------------

# Run from script or as a single line

for d in $(kubectl get deployments -n <namespc>|awk '{print $1}'); do   

   echo "------- App: $d ------"; 

   kubectl get deployment $d -n <namespc> -o yaml > $d.yaml; 

done;

----------------------------------------------------------------------
Minikube Installation
----------------------------------------------------------------------

A basic, easy, single-server kubernetes-like hosting env.

# from the minikube project page
curl -LO https://github.com/kubernetes/minikube/releases/latest/download/minikube-linux-amd64

sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64

ll /usr/local/bin/minikube 

alias kubectl='minikube kubectl --' 

----------------------------------------------------------------------
Minikube Control
----------------------------------------------------------------------

minikube [start|stop|pause|unpause]

----------------------------------------------------------------------

Add-Ons and Services
----------------------------------------------------------------------

minikube addons list
# install
minikube addons enable headlamp

# run addon with name
minikube service headlamp -n headlamp

# run addon
minikube service hello-minikube

# open browser to endpoint, when available from addon
minikube addons open <name>

# run addon when mk starts
minikube start --addons <name1> --addons <name2>

----------------------------------------------------------------------
Deploy Service
----------------------------------------------------------------------

kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0

kubectl expose deployment hello-minikube --type=NodePort --port=8080

kubectl get services hello-minikube

kubectl port-forward service/hello-minikube 7080:8080

----------------------------------------------------------------------
Headlamp Setup
----------------------------------------------------------------------

# install addon - see above
minikube addons enable headlamp

# run addon with name
minikube service headlamp -n headlamp

# get login token
kubectl create token headlamp-admin -n kube-system

# metrics-server helps w/admin
#  uses image registry.k8s.io/metrics-server/metrics-server:v0.7.2
minikube addons enable metrics-server

# See for setup instructions
https://headlamp.dev/docs/latest/installation

~                                                    

New User Setup Script

Adding a new linux user.

----------------------------------------------------------------------
Basic Ubuntu Script
----------------------------------------------------------------------

#!/bin/bash

USERNAME=$1

# Add user and groups
echo "adding user account: $USERNAME"
sudo adduser $USERNAME
sudo usermod -a -G mygroup $USERNAME

# Add sudo (centOS may require wheel group instead)
sudo usermod -a -G sudo $USERNAME

# Download user PKI cert from remote repo - uses server cert
sudo mkdir -p /home/$USERNAME/.ssh
sudo wget -T 90 -t 1 --certificate=/local/server/cert.pem -0 /home/$USERNAME/.ssh/authorized_keys --no-check-certificate https://keyserver/raw/ds/$USERNAME

# Add starter env
sudo cp -p /home/sample/.bashrc /home/$USERNAME/.

# Set perms
sudo chmod -R 700 /home/$USERNAME
sudo chown -R $USERNAME:$USERNAME /home/$USERNAME

Python Packaging

Brief History 

Python builds were traditionally done using:

• distutils - Python built-in, uses requirements.txt to list dependencies, setup.py for configuration
• ISSUES: split of deps and config could cause conflicts, and load order confusion
  
  
• setuptools - extended distutils, added automatic install of dependencies

Other packaging tools:

• wheel - PEP 427 - replaced eggs - build tool with better support for C-library integrations
• creates .pyc files during installation, to match local interpreter
• supported in ip >= 1.4 and setuptools >= 0.8
• PyPI - Python Package Index - online registry of built py libs

among others.

Poetry

Adds main project descritor pyproject.toml to contain project metadata, config, and dependencies.

Some features:

• can declare different dependency sets, i.e. for prod, dev, and test
• creates file poetry.lock to define frozen dependencies

Useful Commands

> Create a New Project

poetry new my-package

> Installing existing Project - exclude some dependencies

poetry install --without test,docs

> Installing existing Project - only include some dependencies

poetry install --only dev

> Install or Fix existing Project - sync local to canonical

poetry install --sync

> Get Info about an Installed Lib

poetry show <package-name>

TOML Example

From https://towardsdatascience.com/ditch-requirements-use-poetry-00a936fe9b6d:

[tool.poetry]
name = "my_project"
version = "0.1.0"
description = "Description of my project"
package-mode = true
authors = [
"My Team <team@my_company.com>",
]

[tool.poetry.dependencies]
python = "~3.10"
fastapi = "^0.111.1"
pydantic = "^2.8.2"
Jinja2 = "^3.1.2"
uvicorn = {extras = ["standard"], version = "^0.30.3"}

[tool.poetry.group.dev.dependencies]
pytest = "^8.2.2"
black = "^24.3.0"
pre-commit = "^3.8.0"
mypy = "^1.11"
flake8 = "^7.1.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.flake8]
max-line-length = 99
extend-ignore = "E203"

Include From Source

Add the following to include a project in development as a dependency.

[[tool.poetry.source]]
name = "my-artifact-registry-repo"
url = "https://europe-west2-python.pkg.dev/my-gcpproject/my-af-repo/simple"
priority = "explicit"