Skip to content

Commit e1db819

Browse files
completed scalability
1 parent a7c38a8 commit e1db819

File tree

10 files changed

+107
-65
lines changed

10 files changed

+107
-65
lines changed

content/backend/web_framework.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -97,8 +97,9 @@ script = 'animation'
9797
<dl>
9898
<dt>Dependencies</dt>
9999
<dd>Don’t use deprecated or vulnerable versions of your dependencies. Use <mark>npm audit fix</mark> to scan your project for vulnerabilities and automatically install any compatible updates to vulnerable dependencies. Alternatively, use <a href="https://snyk.io/"> Snyk.io</a>.</dd><br/>
100-
<dt>Use HTTPS/TLS</dt>
101-
<dd>In HTTPS, the communication protocol is encrypted using Transport Layer Security (TLS) to encrypt data before it is sent from the client to the server, thus preventing some common (and easy) hacks. </dd><br/>
100+
<dt>SSL/TLS Certificate</dt>
101+
<dd>In HTTPS, the communication protocol is encrypted using Transport Layer Security (TLS) to encrypt data before it is sent from the client to the server, thus preventing some common hacks (Man in the middle). HTTPS is based on <b>public/private-key cryptography</b>. This means that there is a key pair: The public key is used for encryption and the secret private key is required for decryption. A website certificate is a public key with a label identifying the owner. when your browser connects to an HTTPS server, the server will answer with its certificate. The browser checks if the certificate is valid and signed by a trusted certification authority. After the verification, the browser extracts the public key and uses it to encrypt information it sends back to the server. The server can decrypt it because the server has the matching private key.
102+
</dd><br/>
102103
<dt>Set security related HTTP headers</dt>
103104
<dd><b>Cross-origin resource sharing (CORS)</b> is a mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served. A web page may freely embed cross-origin images, stylesheets, scripts, iframes, and videos. Certain "cross-domain" requests, notably Ajax requests, are forbidden by default by the same-origin security policy. These have to be set explicitly on the header.</dd><br/>
104105
<dd>There are many other security related headers. Use a middleware function like <a href="https://helmetjs.github.io/">Helmet</a> to set these for you.</dd><br/>
@@ -109,12 +110,11 @@ script = 'animation'
109110
<li><b>Cross-Site Request Forgery (CSRF)</b> - unauthorized commands are transmitted from a user that the web application trusts.</li>
110111
<li><b>Distributed Denial-of-service attack (DDoS)</b> - flooding the server or resource with superfluous requests in an attempt to overload systems and prevent some or all legitimate requests from being fulfilled.</li>
111112
</ul></dd><br/>
112-
<dt>Secure development environments</dt>
113+
<dt>Secure development environments as well</dt>
113114
<dd><ul>
114-
<li><b>Different security concerns than production</b> - For example, in a development environment you may want verbose logging of errors for debugging, while the same behavior can become a security concern in a production environment. And in development, you don’t need to worry about scalability, reliability, and performance, while those concerns become critical in production.
115-
<li><b>Successful key management</b> - involves dealing with the generation, exchange, storage, use, crypto-shredding (destruction) and replacement of keys. It is the more challenging side of cryptography in a sense that it involves aspects of social engineering such as system policy, user training, organizational and departmental interactions, and coordination between all of these elements, in contrast to pure mathematical practices that can be automated.</li>
116-
<li><b>Principle of least privilege</b> - Every process, user, or program must be able to access only the information and resources that are necessary for its legitimate purpose and nothing more.</li>
117-
<li><b>Use HTTPS or a VPN</b> - encrypt data if using unsecured or untrustworthy networks vulnerable to packet sniffing and man-in-the-middle attacks.</li>
115+
<li><b>Successful key management</b> - involves dealing with the generation, exchange, storage, use, destruction and replacement of keys. It is the more challenging side of cryptography as it involves aspects of social engineering, system policy, user training, organizational and departmental interactions, and coordination between all of these elements.</li>
116+
<li><b>Principle of least privilege</b> - Every process, user, or program must be able to access only the information and resources that are necessary for its legitimate purpose and nothing more. This especially applies in teams of developers.</li>
117+
<li><b>Use VPN</b> - encrypt data if using unsecured or untrustworthy networks vulnerable to packet sniffing and man-in-the-middle attacks.</li>
118118
<li><b>Monitor system</b> - Check open ports and unrecognized running processes</li>
119119
</ul></dd><br/>
120120
</dl>

content/devops/scalability.md

Lines changed: 50 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -8,49 +8,72 @@ toc = true
88
script = 'animation'
99
+++
1010

11-
<h3 class="section-head" id="h-Section2"><a href="#h-Section2">Containers</a></h3>
11+
<h3 class="section-head" id="h-Section1"><a href="#h-Section1">Containers</a></h3>
1212
<div class="example">
13+
<p>An operating system feature in which the kernel allows the existence of multiple isolated user-space instances called containers. Containers isolate software from its environment so that it may run uniformly on all machines.</p>
14+
<div class="row">
15+
<div class="col col-6" style="text-align:center">
16+
<figure>
17+
<img alt="Image" src="https://www.javascripter.co/img/devops/container1.PNG" width="85%">
18+
<figcaption>
19+
Container
20+
</figcaption>
21+
</figure>
22+
</div>
23+
<div class="col col-6" style="text-align:center">
24+
<figure>
25+
<img alt="Image" src="https://www.javascripter.co/img/devops/container2.PNG" width="85%">
26+
<figcaption>
27+
Virtual machine
28+
</figcaption>
29+
</figure>
30+
</div>
31+
</div>
1332
<dl>
33+
<dt>Containerization</dt>
34+
<dd>A container is a runtime instance of an image--what the image becomes in memory when executed (an image with state). An <b>image</b> is an executable package that includes everything needed to run an application--the code, a runtime, libraries, environment variables, and configuration files. Containers are a key enabling technology for microservices, providing a lightweight encapsulation of each component so that it is easy to maintain and replicate. </dd>
1435
<dt>Microservices</dt>
15-
<dd>TBD </dd>
36+
<dd>An architectural style that structures an application as a collection of loosely coupled services. The benefit of decomposing an application into different smaller services is that it improves modularity. This makes the application easier to understand, develop, test, and become more resilient to architecture erosion. It parallelizes development by enabling small autonomous teams to develop, deploy and scale their respective services independently. It also allows the architecture of an individual service to emerge through continuous refactoring. </dd>
1637
</dl>
38+
<div style="text-align:center">
39+
<img alt="Image" src="https://www.javascripter.co/img/devops/microservices.png">
40+
</div>
1741
<dl>
1842
<dt>Docker</dt>
19-
<dd>Ports </dd>
43+
<dd>A platform for developers and sysadmins to develop, deploy, and run applications with containers. </dd><br/>
44+
<dd>The Docker <b>Client</b> runs commands like <mark>docker build</mark> or <mark>docker run</mark>. The Docker <b>Daemon</b> (aka Host or Engine) makes the system calls to create, operate and manage containers. Many of the configurations in Docker image will from Docker <b>Registry</b> where it will be downloaded. </dd>
45+
<div style="text-align:center">
46+
<img alt="Image" src="https://www.javascripter.co/img/devops/docker.png" width ="85%">
47+
</div><br/>
48+
<dd><b>Swarm</b> is a native clustering tool for Docker. Swarm pools together several Docker hosts and exposes them as a single virtual Docker host. In Swarm mode, there are two types of nodes: Managers and workers. Manager nodes maintain cluster state and schedule services. Worker nodes execute containers. Clustering is an important feature for container technology, because it creates a cooperative group of systems that can provide redundancy through failover and scalability through automation.</dd>
2049
</dl>
50+
<div style="text-align:center">
51+
<img alt="Image" src="https://www.javascripter.co/img/devops/swarm.png">
52+
</div><br/>
53+
<dd><b>AWS ECS</b> and <b>Kubernetes</b> are two alternate services for maintaining and coordinating clusters of container. <i>Their functionlity is similar</i>.</dd>
2154
</div>
2255
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
2356

24-
<h3 class="section-head" id="h-Section3"><a href="#h-Section3">Load balancing</a></h3>
25-
<div class="example">
26-
<dl>
27-
<dt>ALB vs ELB</dt>
28-
<dd>TBD </dd>
29-
</dl>
30-
<dl>
31-
<dt>Autoscaling</dt>
32-
<dd>TBD </dd>
33-
</dl>
34-
</div>
35-
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
36-
37-
<h3 class="section-head" id="h-Section4"><a href="#h-Section4">Web performance</a></h3>
57+
<h3 class="section-head" id="h-Section2"><a href="#h-Section2">Web performance</a></h3>
3858
<div class="example">
3959
<dl>
60+
<dt>Load balancing and Autoscaling</dt>
61+
<dd>A load balancer is a device that acts as a reverse proxy and distributes network or application traffic across a number of servers. Autoscaling is when computational resources in a server farm, typically measured in terms of the number of active servers, scales automatically based on the load on the farm. When using containers, orchestration services <ins>Kubernetes and Amazon ECS handle load balancing and autoscaling</ins> for you.</dd><br/>
4062
<dt>CDN</dt>
41-
<dd>Static files </dd>
42-
</dl>
43-
<dl>
63+
<dd>A content delivery network (CDN) refers to a geographically distributed group of servers which work together to provide fast delivery of Internet content. A CDN allows for the quick transfer of static assets needed for loading Internet content including HTML pages, javascript files, stylesheets, images, and videos. Using a CDN improves website load times, reduces bandwidth costs, increases content availability and redundancy, and improves website security (e.g. DDoS mitigation).</dd><br/>
64+
<div style="text-align:center">
65+
<img alt="Image" src="https://www.javascripter.co/img/devops/cdn.png">
66+
</div>
4467
<dt>Caching</dt>
45-
<dd>Service workers, Redis </dd>
46-
</dl>
47-
<dl>
68+
<dd>A <b>cache</b> A hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. A CDN is an example of a cache when a user retrieves assets from an edge location rather than a distant origin server. Web browsers also uses caches that store static data rather than re-retrieve them. The timeframe for how long these items stay cached can be set by a <b>cache header</b> or <b>service worker</b>. For databases, <b>Redis</b> can be used to intercept requests and send cached data rather than querying the database additional times for the same data.</dd><br/>
4869
<dt>Data transfer</dt>
49-
<dd>GZIP, QUIC, streams </dd>
50-
</dl>
51-
<dl>
70+
<dd><b>QUIC</b> - An experimental transport layer network protocol that is built on top of UDP. It handles error handling without the handshaking that the higher latency protocol TCP has.</dd>
71+
<dd><b>Compression</b> - compress all static assets. Also make sure your server uses <b>gzip</b> - a file format and a software application used for file compression and decompression.<br/></dd>
72+
<dd><b>Lazy loading</b> - Transfer data to the client as it is needed like a stream, rather than having them wait for the entire bundle to complete downloading.
73+
</dd><br/>
5274
<dt>Language selection</dt>
53-
<dd> Java, webassembly </dd>
75+
<dd><b>Java</b> - For some tasks a different language might be necessary for performance gains. Java is an excellent language for computationally intensive operations. IO operations, on the other hand, might be better served using Node.js.</dd>
76+
<dd><b>WebAssembly</b> - a web standard that defines a binary format and a corresponding assembly-like text format for executable code in Web pages. It is meant to enable executing code nearly as quickly as running native machine code. Many believe WebAssembly could replace JavaScript entirely in the distant future.</dd>
5477
</dl>
5578
</div>
5679
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>

content/devops/testing.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -82,10 +82,20 @@ Note that code written without a test-first approach is often very hard to test!
8282
<li><a href="https://developers.google.com/web/tools/chrome-devtools/">Official Documentation</a></li>
8383
</ul>
8484
</dd>
85-
<dt>Server-side debugging</dt>
86-
<dd>Logs - cloudwatch</dd>
87-
<dd>Instance health check</dd>
88-
<dd>Sentry.io</dd>
85+
<dt>Error tracking</dt>
86+
<dd><b>Logging</b> - the practice of storing large volumes of computer generated audit logs for later analysis. The creation of logs is dependent on the goals of the application. In addition, there are middleware packages like <a href="https://github.com/expressjs/morgan">Morgan</a> that can automate some of these processes. For reporting, modern logging systems use <a href="http://localhost:1313/backend/databases/#h-Section4">Elasticsearch</a> to visualize logs and metrics using string analysis. <br/>
87+
<ins>Logging can also be used for:</ins>
88+
<ul>
89+
<li>Compliance with security policies</li>
90+
<li>Compliance with audit or regulation</li>
91+
<li>System troubleshooting</li>
92+
<li>Forensics (during investigations or in response to subpoena)</li>
93+
<li>Security incident response</li>
94+
<li>Understanding online user behavior</li>
95+
</ul>
96+
</dd>
97+
<dt>Monitoring</dt>
98+
<dd>It is important to monitor systems and send automated notifications to respond to system-wide performance changes. On AWS, <b>CloudWatch</b> provides a unified view of AWS resources, applications and services that run on AWS. You can set alarms, visualize logs and metrics, take automated actions, configure health checks, troubleshoot issues, and discover insights to optimize your applications, and ensure they are running smoothly. Another service essential monitoring system health is <a href="https://sentry.io">Sentry.io</a> which provides client-side error notifications.</dd>
8999
</dl>
90100
</div>
91101
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>

content/latest/big_data_blockchain.md renamed to content/latest/big_data.md

Lines changed: 0 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -34,30 +34,3 @@ script = 'animation'
3434
</dl>
3535
</div>
3636
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
37-
38-
<h3 class="section-head" id="h-Section4"><a href="#h-Section4">Intro</a></h3>
39-
<div class="example">
40-
<dl>
41-
<dt>Use cases, mining, blocks/hash, nodes</dt>
42-
<dd>TBD </dd>
43-
</dl>
44-
</div>
45-
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
46-
47-
<h3 class="section-head" id="h-Section5"><a href="#h-Section5">Smart contracts</a></h3>
48-
<div class="example">
49-
<dl>
50-
<dt>EVM</dt>
51-
<dd>TBD </dd>
52-
</dl>
53-
</div>
54-
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
55-
56-
<h3 class="section-head" id="h-Section6"><a href="#h-Section6">Solidity</a></h3>
57-
<div class="example">
58-
<dl>
59-
<dt>Commonly used</dt>
60-
<dd>TBD </dd>
61-
</dl>
62-
</div>
63-
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>

content/latest/blockchain.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
+++
2+
description = "Blocks, mining, nodes, smart contracts"
3+
title = "Blockchain"
4+
draft = false
5+
weight = 300
6+
bref="Big data is a term used to refer to data sets that are too large or complex for traditional data-processing application software to adequately deal with"
7+
toc = true
8+
script = 'animation'
9+
+++
10+
11+
<h3 class="section-head" id="h-Section4"><a href="#h-Section4">Intro</a></h3>
12+
<div class="example">
13+
<dl>
14+
<dt>Use cases, mining, blocks/hash, nodes</dt>
15+
<dd>TBD </dd>
16+
</dl>
17+
</div>
18+
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
19+
20+
<h3 class="section-head" id="h-Section5"><a href="#h-Section5">Smart contracts</a></h3>
21+
<div class="example">
22+
<dl>
23+
<dt>EVM</dt>
24+
<dd>TBD </dd>
25+
</dl>
26+
</div>
27+
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>
28+
29+
<h3 class="section-head" id="h-Section6"><a href="#h-Section6">Solidity</a></h3>
30+
<div class="example">
31+
<dl>
32+
<dt>Commonly used</dt>
33+
<dd>TBD </dd>
34+
</dl>
35+
</div>
36+
<div style="text-align:right"> <a href="#top">&#8593; Top</a></div>

static/img/devops/cdn.png

25 KB
Loading

static/img/devops/container1.PNG

14.2 KB
Loading

static/img/devops/container2.PNG

15.8 KB
Loading
14.5 KB
Loading

static/img/devops/swarm.png

96.5 KB
Loading

0 commit comments

Comments
 (0)