03-Solution-Application-Failure

Practice Test - Application Failure

• Take me to Practice Test of the Application Failure

Solution

1. A simple 2 tier application is deployed in the alpha namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.

   Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.

   1. Click on the App button at the top of the terminal. Observe the error message.

   It is telling us

   Name does not resolve
   

   This is a DNS lookup issue. Coredns does not know any service called mysql-service, however the architecture diagram says that there should be this service.

   1. Examine services in the alpha namespace

      kubectl get service -n alpha

      We see there is a service mysql so the likelihood is that the service is deployed with incorrect name

   2. Fix it

      Note that you cannot use kubectl edit to change a resource name.

      kubectl get service -n alpha mysql -o yaml > service.yaml
      vi service.yaml
      

      apiVersion: v1
      kind: Service
      metadata:
        creationTimestamp: "2023-10-14T11:40:30Z"
        name: mysql           # <- Edit this to mysql-service
        namespace: alpha
        resourceVersion: "824"
        uid: d9a85021-547a-4a39-b254-0480830eab6a
      spec:

   3. Delete and recreate service

      kubectl delete svc mysql -n alpha
      kubectl create -f service.yaml

2. The same 2 tier application is deployed in the beta namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.

   Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.

   1. Click on the App button at the top of the terminal. Observe the error message.

      It is telling us

      Can't connect to MySQL server on 'mysql-service:3306'
      

      Which suggests that the service exists but the port settings may be incorrect.

   2. Edit the service

      kubectl edit service mysql-service -n beta
      

      apiVersion: v1
      kind: Service
      metadata:
        creationTimestamp: "2023-10-14T11:54:04Z"
        name: mysql-service
        namespace: beta
        resourceVersion: "1166"
        uid: d6b07c71-5c49-4118-849a-0b12dd382597
      spec:
        clusterIP: 10.43.42.85
        clusterIPs:
        - 10.43.42.85
        internalTrafficPolicy: Cluster
        ipFamilies:
        - IPv4
        ipFamilyPolicy: SingleStack
        ports:
        - port: 3306        # <- Correct
          protocol: TCP
          targetPort: 8080  # <- Incorrect, should also be 3306
        selector:
          name: mysql
        sessionAffinity: None
        type: ClusterIP

      The mysql database server listens on port 3306 meaning that is the port that its pod will expose, and where targetPort should be pointing. We can verify that like this

      kubectl get po -n beta mysql -o jsonpath='{.spec.containers[*].ports[*].containerPort}'
      

      or simply get the yaml for the pod.

3. The same 2 tier application is deployed in the gamma namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed or unresponsive. Troubleshoot and fix the issue.

   Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.

   1. Click on the App button at the top of the terminal. Observe the error message.

      It is telling us

       Can't connect to MySQL server on 'mysql-service:3306' (111 Connection refused)
      

      Which suggests that the service exists but there is something wrong with it.

   2. Edit the service

      kubectl edit service mysql-service -n gamma
      

      This time the name is correct, as are both the ports. So perhaps the pod selector is the issue

      Open an additional terminal (+ button above terminal) so as not to have to quit vi now, and run

      kubectl get pods -n gamma --show-labels
      

      Note the labels. Always ignore pod-template-hash label. It is used internally by kubernetes.

      name=webapp-mysql,pod-template-hash=5456999f7b
      

      That doesn’t match with sql00001 in the service selector. Switch back to your vi session in Terminal 1 and fix the selector to use the correct value for the name label.

      apiVersion: v1
      kind: Service
      metadata:
        creationTimestamp: "2023-10-14T12:04:31Z"
        name: mysql-service
        namespace: gamma
        resourceVersion: "1441"
        uid: 872dfbcd-0b1d-4292-85f6-803d62d05b0a
      spec:
        clusterIP: 10.43.21.189
        clusterIPs:
        - 10.43.21.189
        internalTrafficPolicy: Cluster
        ipFamilies:
        - IPv4
        ipFamilyPolicy: SingleStack
        ports:
        - port: 3306
          protocol: TCP
          targetPort: 3306
        selector:
          name: sql00001      # <- Fix this
        sessionAffinity: None
        type: ClusterIP

4. The same 2 tier application is deployed in the delta namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.

   Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.

   1. Click on the App button at the top of the terminal. Observe the error message.

      It is telling us

      Access denied for user 'sql-user'@'10.42.0.16'
      

      So this means the application is using an incorrect mysql user account. The architecture diagram in the question tells you what the correct credentials are.

      So this time the fix is going to be in the application deployment, rather than in the service.

   2. Fix the deployment

      kubectl edit deployment webapp-mysql -n delta
      

      Scroll down to the container’s environment section and fix the user name

        template:
          metadata:
            creationTimestamp: null
            labels:
              name: webapp-mysql
            name: webapp-mysql
          spec:
            containers:
            - env:
              - name: DB_Host
                value: mysql-service
              - name: DB_User
                value: sql-user   # <- Fix this
              - name: DB_Password
                value: paswrd
              image: mmumshad/simple-webapp-mysql
              imagePullPolicy: Always
              name: webapp-mysql
       

5. The same 2 tier application is deployed in the epsilon namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.

   The question indicates there are two issues that need fixing.

   1. Click on the App button at the top of the terminal. Observe the error message.

      It is telling us

      Access denied for user 'sql-user'@'10.42.0.16'
      

      So it is another authentication issue.

   2. Check the deployment

      Vaildate the environment against the values provided in the architecture diagram. Looks like the same issue as previous question, so fix that the same way.

   3. Wait! We fixed that, but it’s still not working!

      Check the mysql pod, since mysql also needs some credential information on its end.

      We cannot use kubectl edit to change values of a standalone POD.

      kubectl get pod -n epsilon mysql -o yaml > mysql.yaml
      vi mysql.yaml
      

      apiVersion: v1
      kind: Pod
      metadata:
        creationTimestamp: "2023-10-14T12:21:51Z"
        labels:
          name: mysql
        name: mysql
        namespace: epsilon
        resourceVersion: "2002"
        uid: f369770b-be24-43c1-b754-d7e1396d6952
      spec:
        containers:
        - env:
          - name: MYSQL_ROOT_PASSWORD
            value: passwooooorrddd    # <- Fix this
          image: mysql:5.6
          imagePullPolicy: IfNotPresent
          name: mysql
          ports:

   4. Recreate the pod

      kubectl replace --force -f mysql.yaml
      

6. The same 2 tier application is deployed in the zeta namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.

   The question indicates there are three issues to fix.

   1. Click on the App button at the top of the terminal. Observe the error message.

      We get a 502 Bad Gateway error.

      This is indicative that the lab display infrastructure cannot connect to the service it’s supposed to. Examine the URL in the browser address bar

      30081-port-1795f98fde814933.labs.kodekloud.com/
      

      On KodeKloud labs, the 30081-port part indicates a node port it’s trying to connect to. Note also that the infrastructure diagram states that 30081 should be the web-service nodeport.

   2. Examine web-service since that’s how we view the app.

      kubectl edit service -n zeta web-service
      

      apiVersion: v1
      kind: Service
      metadata:
        creationTimestamp: "2023-10-14T13:27:26Z"
        name: web-service
        namespace: zeta
        resourceVersion: "1530"
        uid: 9b655dda-675a-43ae-80e4-deadb3a38179
      spec:
        clusterIP: 10.43.48.98
        clusterIPs:
        - 10.43.48.98
        externalTrafficPolicy: Cluster
        internalTrafficPolicy: Cluster
        ipFamilies:
        - IPv4
        ipFamilyPolicy: SingleStack
        ports:
        - nodePort: 30088  # <- Edit this
          port: 8080
          protocol: TCP
          targetPort: 8080
        selector:
          name: webapp-mysql
        sessionAffinity: None
        type: NodePort

   3. Retry the app in the browser

      We have seen this message before! Fix it as per above, using namesapce zeta.

   4. Retry the app in the browser

      We have also seen this message before! Fix it as per above, using namesapce zeta.