Sunday, July 17, 2022

NFR : None Functional Requirement in Software Engineering or Application Development : Part 4: Extensibility, Observability, Maintainability

 In this post, we are going to discuss about the remaining 3 NFRs like Extensibility, Observability, Maintainability. The link of all of my posts related with NFR are as follows:

Links:

Extensibility: Extensibility is a measure of the ability to extend an application and the level of effort required to implement the extension. It's components are like:

  1. Flexibility: It defines how flexible your application is towards accepting business growing needs and new functionality changes. We can make our application more flexible by applying OOPS concepts, Dependence Injection, Applying Microservice/ Microviews architecture etc.
  2. Configurability: It measures the extent of configuration used in your application. We should be able to add or hide the functionality from an application by configuration. There should not be any code related with any 3rd party application or vendor. It should be in config files. So that, we can replace the 3rd party application by changing in config files and very minimal change in code. This way it will require less testing and application will be ready very quickly with less effort and minimum impact.
  3. Customizability: It is very closely related with configurability. If application is highly configurable then it will be highly customizable. The vendor or client specific functionality in your application should be in plug-in-play mode.  
  4. Upgradeability: It defines how easily your application's different components can be upgraded. Generally, we should use the built-in features of the technologies in our application so that we can upgrade our technology and all the built-in features will be upgraded automatically and easily. If we are using any 3rd party vendor's feature its difficult to upgrade as the upgraded feature may not support our application's underlying technology.
  5. Integrity: The application should be well integrated with other required business system. For example, your application should be integrated with analytic tool, dataware house etc. It should be capable to sharing the correct reports in terms of csv or excel with other business system. It should be easily integrated with any new incoming business component or vertical by applying some configuration changes and minimal code changes.
Observability: Observability is all about data exposure and easy access to information required to find issues when the communications fail, internal events do not occur as expected or events occur when they shouldn’t. The observability of your application can be enhanced by proper logging and error handling at every decision point or at the point of communication between different APIs and 3rd party API. We can create some Dashboard to display different metrics of our application. For these purpose, we can use tools like Grafana, Dyanatrace etc. We should also enable dashboard to visualize the logs like Kibana. In case of AWS, we can CloudWatch, Flowlogs for these purpose.

Maintainability: It refers the ease with which we can understand, repair and improve the code of our application. The more maintainable our application is the more easily we can fix bugs, enhance our application's functionalities and fixing the security issues. While performing maintenance activities we can make 4 types of changes to our application:
  1. Corrective: It is related with fixing the bugs and issues in our application. Application code should be well segregated in different files and folders according to its purpose so that required code fixes can be applied easily at the required place.
  2. Adaptive: Our application should be developed in such a way that it can easily accept new functionalities. If needed we can easily removed the deprecated or unwanted functionalities. It should be able to accept the updated libraries, DB, operating system etc.
  3. Perfective: This functionality includes the code changes to improve the application performance.
  4. Preventive: It is associated the required changes in application for enhancing security. Upgrading security is an ongoing and never end work. So, our application should always be ready to accept such security related changes.


NFR : None Functional Requirement in Software Engineering or Application Development : Part 3: Application Security

 In my previous posts, I have discussed about NFR definition, its requirement and 3 important NFRs like Scalability, Performance and Testability. Below are the links of all of my posts related with NFR:

Links:

Today, I am going to discuss about a very important NFR that is Application Security. Application security is an ongoing and never ending work. Every day, new security threats comes in cyber space and we along with Business team will have to keep eye open on these security threats and if it is going to impact our application, we should take preventive measures as soon as possible. Generally, application security is the responsibility of not only the Development team but it depends on all the participants of the application like: 

  • Business team
  • Development team
  • End users
  • Hosting environment provider
  • Network provider
Guidelines and conventions has been provided for each participants over internet and if every participants follows performs their roles very cautiously then only we can achieve the application security goal. Normally, it has been found that most of the application security issues comes either due to security loop holes present in application development side or from the End user side. The security issues from End user side can be minimized by increasing the awareness about secure use of the application among End users. This is completely separate set of efforts that application owner's Customer facing team should put to make their customer aware about secure use of application as much as possible. Here, I am not concentrating about customer awareness(although it is backbone of any application security). Here, I will discuss about the Development teams roles regarding implementation of security features in an application.

I have developed several application in Node js and applied below security features. But, these are applicable to any public facing web application developed in any technology.

  1. CORS vulnerability: It is abbreviated as Cross Origin Resource Sharing. Here, generally, our application exposes some APIs to share some data to some particular 3rd party API. So, we should keep a check that whether our exposed APIs are communicating with valid 3rd party APIs or some malicious services. This can be achieved by defining a White List array of 3rd party APIs or domain that will be allowed to interact with our application and rest all will be denied. There are several ways of implementing it. In Node Js, we can use CORS NPM package to implement it. If we are deploying our application in cloud like AWS, these cloud providers different security features like NACL, SecurityGroup
  2. CSRF vulnerability: It is abbreviated as Cross Site Request Forgery. Here, some malicious 3rd party API tries to send a malicious request to your application in such a way that your application assumes that the request is coming from a valid authenticated user. In Node JS application, this vulnerability can be restricted by using CSURF NPM package. Using this package we create a csurf token at Node JS side in a GET method at the very first time when request comes and then this token is sent to client in response body. Now, client should get this token from response body and set it in request header every time it is sending any request to Node JS. On Node JS side, we can define a middleware that will take this token from response header and try to match it with the token generated by Node JS. If it is matching then process the request and if it is not matching then return error response.
  3. Request payload size limit: Defining request payload max size limit on API side is very important. Generally, Node JS API reads the JSON request payload in synchronous mode and if the request payload size is very large then your application might hang or will become very slow. It should be checked before processing any request on nodejs side. If using Express framework, this size can be defined as app.use(express.json({ limit: 10 })). Here request payload size limit is defined as 10 BYTES. The default request payload size limit in EXPRESS is 100KB. I have created a post on it. Its URL is: Node JS application security
  4. File upload size limit: Define max file size limit if your application supports file upload. It can be defined by using Multer package like  multer({limits: { fileSize: maxSize }}). Here, maxSize is in BYTES.
  5. Input data sanitization: Generally, the fraud like SQL injection and Javascript injections are done by input form data. This can be minimized by sanitizing the input data on API side before using it. Several NPM packages are available to sanitize input data like express-validator. In any database interaction, we should use parametrized query or any ORM library to minimize the chances of SQL injection. 
  6. Restrict XSS(Cross site scripting) attack: This is very much similar to the security issue discussed just above under the heading "Input data sanitization". Here, along with above sanitization measures we can use XSS NPM package to read data from request object.
  7. Use HTTPS instead of HTTP
  8. Authorization: We should follow the least privilege's mechanism with no previlege as Default. This can be implemented either by our own logic by creating tables where we can keep group of users with roles at group level and check the user's group and role before processing any request. Some available NPM package supporting it are @casl/ability, passport etc.
  9. Authentication: Use some well established 3rd party authentication mechanism instead of developing your own from scratch. The example of 3rd party authentication mechanism are like Social media authentication, Authentication using Azure or AWS authentication services. After successful authentication, we should create a JWT token by putting required data in token by encrypting it using Encryption algorithm provided by JWT. This token should be signed and sent to client. It should be properly validated before processing every request.
  10. If any important data is coming from UI to API side it must be encrypted. The encryption key must be there in config file in UI. Same key must be used in API side to decrypt it.
  11. Proper logging: Proper logging should be maintained for each request keeping its user, IP information, timestamp, operation success or failure information etc.
  12. OWASP top ten: The OWASP is an organization that provides the list of top 10 security vulnerabilities of application every year. The details can be found on https://owasp.org/Top10 These top 10 vulnerabilities must be taken care in our application at any cost. The top 10 security vulnerabilities for the year 2021 are as follows:
  • A01:2021-Broken Access Control: It occurs if we provide unwanted or extra privilege's to an user. It will be restricted by the ways discussed under the heading Authorization above with least privilege's.
  • A02:2021-Cryptographic Failures: It occurs due to missing required encryption or due to encryption by deprecated or old libraries. It can be restricted by the ways discussed in point (10). The only difference is that the point(10) is talking about data in transit but this is talking about data in transit as well as data in rest. Here, we should use latest cryptographic libraries and always avoid deprecated library like md5 and SHA1. 
  • A03 Injection: It can be restricted by sanitizing request data and using parametrized queries in DB interaction as discussed in point(5 and 6)
  • A04:2021 – Insecure Design: It is mainly related with Project Design. If project is not designed properly, even best coding practices can not rectify it. There are several tricks to minimize this flaws like using Limiting resource consumption by user or service, writing proper and sufficient unit test and integration test cases, using well established NPM packages. 
  • A05 Security Misconfiguration: Security is very vast and as a software engineer we have to constantly evaluate and upgrade our security feature in our application. We will have to keep verifying all the Cloud security features like SecurityGroup, NACL(Network Access Control Layer), IAM(Identity access management) policy etc. We should keep our 3rd party libs and NPM libs up to date. We should always try to move to the latest version of the programming language and DB if possible.
  • A06:2021 – Vulnerable and Outdated Components: This can be mainly rectified by using the latest version of our programming language, DB, NPM package and any 3rd party APIs.
  • A07:2021 – Identification and Authentication Failures: It mainly occurs if we are not able to keep our identity safe or using very week identity credentials like password or forget password question. To rectify it we should force user to make strong password and keep them properly encrypted in transit as well as in rest. Design your application to make it safe from Brute force attacks or automated attacks. If any crucial API is failing for 3 or more times for a particular user then block him for 30 min or slow down the processing for him. This kind of design will help in restricting automated attacks.
  • A08:2021 – Software and Data Integrity Failures: It is mainly associated with the integrity failure or vulnerabilities related with the 3rd party APIs or NPM packages or CDN we are using in our application. So, if we are using any 3rd Party API, it should be interacted with HTTPS protocol along with proper validation of authentication and authorization tokens. We should use well established NPM package. 
  • A09:2021 – Security Logging and Monitoring Failures: It is associated with missing required logging and monitoring the logs regularly. It can be prevented by using proper logging in every request and setting up some Alarm or event like CloudWatch alarm in case of AWS to get notification via email or SMS about continuous failure of a particular API. In such cases we can validate logs to identify the cause of failure and to get the clue of any security attacks.
  • A10:2021 – Server-Side Request Forgery (SSRF): It mainly occurs by redirecting user to a different URL. It can be restricted by adding code for properly validating the domain name and port of the URL to ensure whether the domain, host and port are coming in our white list URL before redirecting user to that URL.
Here, I have discussed the minimum required measures that we must take in our application development from security perspective. But, it is not a vast list of security features. There are several more tips and tricks that we can apply in our application. Application security implementation is an ongoing process and development team as well as Business team will have to keep working on it. Its not an one time activity.

In my next post, I will discuss about few other NFR topics.

Saturday, July 16, 2022

NFR : None Functional Requirement in Software Engineering or Application Development : Part 2: Scalability, Performance, Testability

 In my previous post, I had discussed about below points like:

  1. What is NFR
  2. What information should be gathered by Development team from Business team with respect to NFR
  3. Listed down few important NFRs
Link of my NFR related posts:

In the continuation of my previous post about NFR, I am going to discuss about 3 NFR like Scalability, Performance, Testability

Scalability:
Scalability is the measure of a system's ability to increase or decrease in performance and cost in response to changes in application and system processing demands. There are 3 important types scalability:
  1. Horizontal Scaling: New server is added instead of upgrading the existing server.
  2. Vertical Scaling: Existing server is upgraded based by increasing CPU, RAM, Processor
  3. Diagonal Scaling: Here, both horizontal and vertical scaling are done.
Below are few of the ways of implementing scalability in your application:
  1. Setup an alarm like CloudWatch Alarm in AWS that triggers an alarm or event when CPU utilization reaches a thresh hold value
  2. Setup an autoscaling by using cloud autoscale functionality or Kubernates
  3. Load Balancing: Define load balancer to equally distribute load on several servers
  4. Enable clustering if your application technology supports. E.g. if your application is using NodeJs for API creation, we can easily enable clustering as discussed in my below posts like:
            Cluster in Node Js application

Performance:
It represents the application's throughput time. Lower the throughput time, higher is the performance. Below are few of the ways to enhance your application's performance:
  1. Setup proper load balancer
  2. Use auto-scaling to enhance performance based on CloudWatch alarm
  3. Setup caching like Reddish or memcache
  4. Flush the cache data at certain intervals based on frequency of write operation in the application and how early to present the latest data to customers
  5. Always use Asynchronous operations where synchronous behavior is not required
  6. Follow all coding conventions and guidelines
  7. Follow all best practices of DB like SQL query optimization, using Views, Indexes, Stored Procedures and SQL functions where ever required.
  8. Use some code validation tools like Sonar or Eslint
  9. Try to use latest stable and suitable version of libs like NPM packages
Testability:
In software, testability refers to the degree that any module, requirements, subsystem or other component of your architecture can be verified as satisfactory or not. High testability means it is easy to find and isolate faults as part of your team's regular testing process. Our application must have following criteria to be more testable:
  1. Controllability: The degree to which it is possible to control the state of the component under test (CUT) as required for testing.
  2. Observability: The degree to which it is possible to observe (intermediate and final) test results.
  3. Isolateability: The degree to which the component under test (CUT) can be tested in isolation.
  4. Separation of concerns: The degree to which the component under test has a single, well defined responsibility
  5. Understandability: The degree to which the component under test is documented or self-explaining.
  6. Automatability: The degree to which it is possible to automate testing of the component under test.
  7. Heterogeneity: The degree to which the use of diverse technologies requires to use diverse test methods and tools in parallel.
In my next post, I will discuss about NFR related with Application security.