Frequently Asked Questions (FAQ)#

How do I add my custom Generic Base App (GBA) to the deployment?#

You can use the application chart to add your GBAs to a deployment. Please follow the instructions in the chart README.

I do not find any of my custom objects (roles, classes, …) from my GBA in the system. Is there an install log file that I can check?#

Yes. You can either check the log of the application job with

kubectl logs -l nplus/instance=sbs,nplus/component=application

or you can check the log at /conf/<instance>/application/10init.log from the environment toolbox.

Please check out the chart README for more information.

Please note, that the job/pod is automatically removed shortly after app installation, so the kubectl logs command might not find the ressource any more.

Network Policies#

Kubernetes CNI supports the use of NetworkPolicy resources. Every resource, that has a NetworkPolicy attached is monitored by a compatible CNI driver such as Calico oder Cilium and Network Filter Rules are implemented.

By this means, one pod can only communicate with other pods, if a network rule has explicely been applied.

nplus supports NetworkPolicies by the following control structures:

security.cni. (on component, instance or environment level)

  • defaultIngressPolicy can be set to deny, allow or none. deny will drop all undefined inbound packages, allow will forward all undefined inbound packages If not defined, the Policy will not be created.
  • defaultEgressPolicy can be set to deny, allow or none. deny will drop all undefined outbound packages, allow will forward all undefined outbound packages If not defined, the Policy will not be created.
  • createNetworkPolicy toggles the policy creation in general

For larger projects, it is likely to have a Central Services Instance that hold e.g. the Administrator and the Monitoring Console. If these services are in the same namespace and within the same instance, nothing need to be done (default).

However, if you use Central Services you can define the Namespace and the Instance of these services in order to have NetworkPolicies created for inter-namespace and inter-instance traffic.

  • administratorNamespace
  • administratorInstance
  • monitoringNamespace
  • monitoringInstance
  • pamNamespace
  • pamInstance

If you use a centralized Storage Layer and Rendition Server, you will have to apply extra Policies to allow access. Please remember to write ingress and egress rules.

Example:

global:
  environment:
    security:
      cni:
        defaultIngressPolicy: deny
        defaultEgressPolicy: deny
        createNetworkPolicy: true

How can I use snc in NAPPL to access my SAP System?#

To use snc in NAPPL, you need to

  1. Enable it in NAPPL (nappl.snc.enabled: true)
  2. Add the IP Range of your SAP Systems to allow egress access (nappl.snc.sapIpRange: "0.0.0.0/0")
  3. Copy the snc files to the nplus environment (kubectl cp snc nplus-toolbox-0:/conf/pool)

Please find more information in the nappl chart README

How can I use extra fonts for rendition or OCR?#

Extra fonts, like the mscorefonts can be installed by copying them into the nplus environment. The fonts are then automatically applied to all rendition Server and Application Layer components within all nplus Instances within this environment.

To copy fonts to the pool, use

kubectl cp test/fonts nplus-toolbox-0:/conf/pool

This copies the local fonts directory to the environment pool. The target is pool/fonts, where all extra fonts must reside. This is then picked up by the components.

How can I completely remove any trace of nplus from my cluster?#

  1. Remove all nplus Instances from your nplus Environment:

If you installed with helm:

helm uninstall myInstance

If you installed using Argo:

helm install myInstance-argo

or whatever the name of your instance is.

If you installed by kubectl:

kubectl delefe -f myInstance.yaml

Do this for all instances.

  1. Remove the nplus Environment from the Kubernetes Namespace

if installed by helm:

helm uninstall <name>

where name is the name you used when installing

  1. Remove the nplus Cluster from the Kubernetes Cluster

if installed by helm:

helm uninstall <name>

where name is the name you used when installing

I would like to connect to the environment dav server to access the config files#

You can access the nplus Environment conf dav server either

  • through an ingress, if you enable it. But you might want to keep it disabled for security reasons. Instead you can access it
  • via a port forwarding from your local machine, in case you have kubectl access to the cluster:
kubectl port-forward pods/nplus-davserver-0 8080:8080

Then, you can connect to the server via http://localhost:8080/dav

How can I manually delete all Resources belonging to a specific instance?#

To delete everything belonging to a specific instance, you can use:

kubectl delete $(kubectl get svc,sts,deployment,cm,secret,networkpolicy,ing,pvc,certificate,nscale -l nplus/instance=<instance> -o name)

I changed the image tag of nscale Web, but when I apply, the component stays healthy#

Even though it might seem nscale Web would not restart, it actually does. nscale Web is configured as a Rolling Update DeamonSet, so it first creates a new Pod and waits till that is ready. Then it stops the old one.

During the update cycle, the services stays healthy.

Notice, that the Application job (if defined) runs as well. That is, because updating the Web component might require new Snippets etc. to be installed, to nplus is giving the Application the chance to do so.

Can I check out a nappl image?#

Yes, you can:

docker run --rm -it ceyoniq.azurecr.io/release/nscale/application-layer:ubi.9.2.1200.2024052713 /bin/bash

Can I bash into my nappljobs?#

Indeed:

kubectl exec --stdin --tty demo-ha-nappljobs-0 -- /bin/bash

I keep getting errors, that chmod is not allowed on the conf file system#

This might be because you might be using a CIFS / smb shared file system (like Microsoft Azure File). You can switch off all internal chmod commands by setting .Values.global.environment.storage.conf.cifs to true.

We use multiple ingress controllers in different namespaces. How do we set that?#

You can set the ingress class per enviroment, per instance or per component. Component bein the highest priority.

Additionally, you might want to set the namespace of your controller to allow ingress traffic from that namespace to the pods. Since you probably have multiple namespaces, this is a comma separated list:

# Set Ingress namespace per component
ingress:
  namespace: "nginx-ingress"

or

# Set Ingress namespaces for all instances in an environment
global:
  environment:
    ingress:
      namespace: "ingress, kube-system, external-ingress, internal-ingress, backup-ingress"

How do I know which tags exist in the registry?#

You can use Skopeo:

skopeo list-tags  docker://ceyoniq.azurecr.io/release/nscale/application-layer

This lists all nappl tags in the registry

We use a forward proxy in our DMZ and have problems with OAuth (or others)#

If you use a forward proxy, such as in a DMZ Scenario, you will probably need to configure your cluster Load Balancer so it forwards the real IP adress of your clients.

In nginx, this is done by the setting use-forwarded-headers which needs to be put into the clusterwide config (this is a global option):

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-load-balancer-microk8s-conf
  namespace: ingress
data:
  use-forwarded-headers: "true"
  proxy-real-ip-cidr: "<Your Reverse Proxy IP>"

Apply this config map to your nginx LB namespace setting the IP Adress CIDR of your DMZ Reverse Proxy.

In the DMZ nginx configuration, make sure you submit all necessary information:

server {
    server_name                 demo.nscale.cloud;
    client_max_body_size        10G;

    proxy_set_header            X-Forwarded-For $remote_addr;
    proxy_set_header            X-Forwarded-Host $host;
    proxy_set_header            X-Forwarded-Proto $scheme;

    if ( $is_bot ) { return 410; }
    location = / { return 301 "/nscale_web"; }
    location = /me { return 301 "/auth/realms/cloud/account"; }
    location /robots.txt { return 200 "User-agent: *\nDisallow: /"; }

    location /nscale_web                                { proxy_pass https://dmz.lan; }
    location ~ ^/(auth/realm|auth/login|auth/resources) { proxy_pass https://centralservices.lan; }
    location /nscalealinst1                             { proxy_pass https://dms.lan; }

    listen 443 ssl;
    ssl_certificate            fullchain.pem;
    ssl_certificate_key        privkey.pem;
}

How yan I set Ressources (CPU / RAM) for the components?#

You can set the ressources in the Values:

resources:
  requests:
    cpu: "100m"         # Minimum 1/10 CPU
    memory: "500Mi"     # Minimum 500 MB 
  limits:
    cpu: "2000m"        # Maximum 2 Cores
    memory: "4096Mi"    # Maximum 4 GB. Java will see this as total.

If you want to set Java Memory Options:

javaOpts:
  javaMinMem: "1024m"
  javaMaxMem: "2048m"

How can I bash into nappl?#

This is an example of how to bash into a nappl, in this case empty-nappl-0:

kubectl exec --stdin --tty empty-nappl-0 -- /bin/bash

How can I set the timezone?#

You can set the timezone per component, instance and/or environment, using the timezone value. Please refer to the component README.md for more information.

How can I use priorityClasses for the components?#

You can use an existing priorityClass by setting priority.className: <your class> on the component, instance or environment.

If you want to have the class created for you, you can set priority.createClass: true.

You can also set the desired value.

Example:

priority:
  className: '{{ .component.fullName }}'
  createClass: true
  value: "1000000"

If you omit the quotes for value, you will end up having a float64 like 1e+06 in your values, which will cause problems.

To forcefully switch off any previously set priority for a specific instance, you can override:

global:
  override:
    priority:

The default is to have no priorityClass at all.

How can I enable and access the Web Administrator?#

To enable the nscale Administrator (Web, aka RapAdmin), you have to first enable the administrator chart in your instance:

components.administrator: true

By default, the Administrator will use the standard Application Layer for login. You can change that by setting

administrator:
  nappl:
    host: '{{ include "nplus.prefix" . }}nappljobs.{{ .Release.Namespace }}'
  waitFor:
    - '-service {{ include "nplus.prefix" . }}nappljobs.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.global.nappl.port }} -timeout 600'

This is an example, where we use multiple Application Layer and one designated Application Layer for Jobs. And we use this nappljobs for administration as well. So the above configuration changes the default and lets the admin client access nappljobs.

If you run the Administrator in another instance (Central Services or something alike), you can also cross namespaces and/or instances here to access multiple tenants if desired. But in that case you might need to add individual networkPolicies to allow access.

Once the Admin Client is running, you can reach it at https://<Your Domain>/rapadm.

I want to use the same domain for my environment and my instance, so the certificates are created twice#

First of all, are you sure you want the same domain? Because the environment ingress is used by admins to access the config by dav or the monitoring data from the operator. You normally would not want that to use the same domain / ingress as the users of your services.

However, if you decide to use the same domain, you can easily switch off certificate generation: Certificates are either generated by an issuer like cert-manager or are self-signed and generated by helm.

  • if .this.ingress.issuer is set, the chart requests this issuer to generate a tls secret with the name .this.ingress.secret
    by creating a certificate resource with the name of the domain .this.ingress.domain
  • else, so no issuer is set, the chart checks wether the flag .this.ingress.createSelfSignedCertificate is set to true and generates a tls secret with the name .this.ingress.secret
  • else, so neither issuer nor createSelfSignedCertificate are set, the charts will not generate anything

After the instance or environment ran through the generation process, the components use the name of the tls secret .this.ingress.secret for their ingresses, in case .this.ingress.enabled is true.

So to cut a long story short:

  1. You better not have the same domain for end users and admins. Please re-consider and try something like

    • admin.my-domain.internal for admin access and
    • my-domain.cloud for public access

  2. If you do want the same domain, you need to switch off the generation process in either the instance or the environment.
    You can still use the same secret. As the environment is deployed before the instance, it might be a good idea to switch off the instance:

    global:
 ingress:
   issuer: null
   createSelfSignedCertificate: false

How can I access my services with a browser?#

Well, that of course depends on

  • which services you enabled
  • if these services gain access through a web interface
  • this access (ingress) is enabled.

You can check like this:

kubectl get ingress -l nplus/instance=<your instance>

Example using the demo-ha example:

% kubectl get ingress -l nplus/instance=demo-ha
NAME                    CLASS    HOSTS                     ADDRESS     PORTS     AGE
demo-ha-administrator   public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-cmis            public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-ilm             public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-mon             public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-nappl           public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-pam             public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-web             public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h
demo-ha-webdav          public   demo-ha.lab.nplus.cloud   127.0.0.1   80, 443   10h

Then, you can drill into an ingress, to get the paths:

kubectl describe ingress <ingress>

You can also get a list of all hosts + paths:

% kubectl get ingress -l nplus/instance=demo-ha -o json 2> /dev/null| jq -r '.items[] | .spec.rules[] | .host as $host | .http.paths[] | ( $host + .path)' | sort | grep -v ^/
demo-ha.lab.nplus.cloud/cmis
demo-ha.lab.nplus.cloud/dav
demo-ha.lab.nplus.cloud/engine.properties
demo-ha.lab.nplus.cloud/index.html
demo-ha.lab.nplus.cloud/modeler
demo-ha.lab.nplus.cloud/nscale_web
demo-ha.lab.nplus.cloud/nscalealinst1
demo-ha.lab.nplus.cloud/nscalealinst1/webb/configuration
demo-ha.lab.nplus.cloud/nscalealinst1/webc/configuration
demo-ha.lab.nplus.cloud/nscalemc
demo-ha.lab.nplus.cloud/rapadm
demo-ha.lab.nplus.cloud/res
demo-ha.lab.nplus.cloud/sap_ilm

I would like to disable the ingress on the operator, but access it through a NodePort Service#

Sure. Just disable the ingress first on your environment deployment:

operator:
  ingress:
    enabled: false

Then add a NodePort Service to access it:

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: nplus-operator-nodeport-access
spec:
  type: NodePort
  selector:
    nplus/component: operator
  ports:
    - port: 8080
      targetPort: 8080
      nodePort: 31976
EOF

Access it:

  • http://<Your Cluster Node IP>:31976/monitoring
  • https://<Your Cluster Node IP>:31977/monitoring

https://10.17.1.31:31977/monitoring/index.html?page=overview

During Desaster Recovery tests we noticed that we cannot change the Document ID in runtime. What should we do?#

You can switch the component (in this case the Storage Layer as you mention the Document ID, but this method work for any component) into Maintenance Mode. Maintenance Mode will

  • start pods without starting the service, providing the possibility to gain access to the container to perform recovery tasks that need to be done offline. In order to do this:
    • All waitFor definitions are ignored
    • All Health Checks are ignored
    • The container starts in idle
    • Application Jobs are disabled

You can put a component, an instance or the whole environment into maintenance.

utils:
  maintenance: true

or global for the instance:

global:
  utils:
    maintenance: true

Why can’t I specify pullSecrets on the waitImage?#

pullSecrets are defined at pod level, not at container level. WaitFor is a container, so it doesn’t have its own pullSecrets but rather takes the pod ones.

We do not want to use argoCD Waves, can we switch it off?#

Yes, just add the following to the values.yaml to globally turn off the argoCD Wave feature:

global:
  utils:
    disableWave: true

Please also see the nowaves example

Out Instances became pretty large with lots of components and multiple team members working on parts of it. Can we somehow slices it into smaller chunks?#

Yes, you can. Simply create multiple Instances with the components you like and then join them all together using a common .instance.group tag. This will open the firewall (Network Policies) to allow traffic within the group / between multiple Instances.

Please see the group example for details

I get frequent DV/DA HID check failures in nstl in my dev Environment#

In the lab / dev environment, you probably quite often throw away the data disk while keeping the conf folder. The default for the DA_HID.DAT is the conf folder, so they do not match any more. You can easily switch the check off:

nstl:
  checkHighestDocId: "0"
nstla:
  checkHighestDocId: "0"
nstlb:
  checkHighestDocId: "0"

if you do this in the environment, you have globally switched all nstl da checks off.

We use the postgres DB for DEV and would like to get a dump. How can we do that?#

You can call pg_dump from the command line. Make sure you have the right password and pod.

kubectl exec --stdin --tty sample-empty-database-0 -- env PGPASSWORD="postgres" pg_dump -U postgres -w nscale > test.dump
Backward Quickstart with Argo CD Operations Forward
nplus - nplus 26.4.1000-1 - © 42i GmbH