7. On demand: scalable architecture of the website
Extensibility (Extensibility):refers to the system with minimal impact on the existing system. The ability to continuously expand or improve functionality. It is the opening and closing principle at the system architecture design level. The architecture design takes into account future functional expansion. When new functions are added to the system, there is no need to modify the structure and code of the existing system.
Scalability (Scalability): refers to the system's ability to enhance (reduce) its own computing and processing capabilities by increasing (decreasing) the scale of its own resources.
A. Build a scalable website architecture
1. The greatest value of a software architect does not lie in how many advanced technologies he has mastered, but in his ability to cut a large system into pieces. The ability to divide it into N low-coupling sub-modules. These sub-modules include horizontal business modules and vertical basic technology modules.
2. The core idea is modularization. On this basis, the coupling between modules is reduced and the reusability of modules is improved.
B. Use distributed message queues to reduce system coupling
1. Event-driven architecture
Event-driven architecture ( Event Driven Architecture): By transmitting event messages between low-coupled modules to maintain loose coupling of modules, and completing inter-module cooperation with the help of event message communication, distributed message queues are commonly used.
#The message queue works using the publish-subscribe model. The message sender publishes the message and one or more message receivers subscribe to the message.
2. Distributed message queue
The queue is a first-in-first-out structure, and the application can access the interface through the remote Use distributed message queues to perform message access operations to achieve distributed asynchronous calls.
The message producer application pushes the message to the message queue server through the remote access interface. The message queue server writes the message to the local memory queue and immediately returns a successful response to the message producer. By. The message queue server searches for the message consumer application that subscribes to the message based on the message subscription list, and sends the messages in the message queue to the message consumer program through the remote communication interface according to the first-in-first-out (FIFO) principle.
Distributed message queues can be very complex. For example, they can support ESB (Enterprise Service Bus) and SOA (Service-Oriented Architecture), or they can also be very simple using MySQL records: The message producer program writes messages into the database as data records, and the message consumer program queries the database and sorts them by the record writing timestamp, thus realizing a de facto distributed message queue.
C. Use distributed services to create a reusable business platform
1. Distributed services decompose system coupling through interfaces , different subsystems make service calls through the interface description of Desert Rose.
2. Problems with the Big Mac system: difficulty in compilation and deployment; difficulty in code branch management; exhaustion of database connections; difficulty in adding new services;
3. Solution
Vertical split: split a large application into multiple small applications
Horizontal split: split the reused business, Deployed independently as distributed services, new businesses only need to call these distributed services and do not need to rely on specific module codes
4. Web Service and enterprise-level distributed services
Disadvantages: bloated registration and discovery mechanism; inefficient XML serialization means; relatively high overhead HTTP remote communication; complex deployment and maintenance means;
5. Large website distribution Requirements and characteristics of distributed services
Load balancing, failover, efficient remote communication, integration of heterogeneous systems, minimal intrusion into applications, version management, real-time monitoring
6. Distributed service framework Design: Thrift, Dubbo
D. Extensible data structure
Designed using ColumnFamily (column family) used in NoSQL databases.
E. Use an open platform to build a website ecosystem
1. The open platform is the interface for internal and external interaction of the website, and the external side needs to face the large number of third-party developers Or, we need to face many business services within the website internally.
2. Architecture: API interface, protocol conversion, security, auditing, routing, process
8. Impregnable: Security architecture of the website
A. Website application attack and defense
1.XSS attack
An attack method is to tamper with web pages, inject malicious HTML scripts, and control the user's browser to perform malicious operations when the user browses the web.
One type of attack is the reflection type. The attacker induces the user to click on a link embedded with a malicious script to achieve the purpose of the attack
Another type of attack is a persistent XSS attack. The hacker submits a request containing a malicious script and saves it in the database of the attacked website. When the user browses the web page, the malicious script is included in the normal page, achieving the purpose of the attack. Purpose. It is often used in web applications such as forums and blogs.
Preventative measures include disinfecting and filtering dangerous characters, while prohibiting page JS from accessing Cookies with HttpOnly attributes
2. Injection attack
It is divided into SQL injection and OS injection
6. Website security vulnerability scanning
B. Information encryption technology and key security management1. One-way hash encryption: md5, sha, etc., add salt
2. Symmetric encryption: DES algorithm, RC algorithm, etc., use the same key for encryption
3. Asymmetric encryption: RSA Algorithm
4. Key security management
1. Text matching: solving the problem of sensitive word filtering
3. Blacklist: Hash table, Bloom filter
D. E-commerce risk control1. Risks: account risk, buyer risk, seller risk, transaction risk
2.Risk control
1.LAMP->JAVA/ORACLE->MySQL/ NoSQL
2. Business drives the continuous progress of technology
10. Wikipedia’s high-performance architecture design analysisA.Wikipedia website as a whole Architecture:
LAMP open source products, GeoDNS, LVS, Squid, Lighttpd, PHP, Memcached, Lucene, MySQL
B.Wikipedia performance optimization strategy1. Front-end performance optimization
3. Back-end performance optimization:
Cache
11. High-availability architecture design analysis of Doris, a massive distributed storage system
For a data storage system, high availability means: high availability Available services, highly reliable data
A. High-availability architecture of distributed storage system
1. Redundancy: server hot backup, multiple data storage
2. Overall system division:
Application server: The client of the storage system initiates a data operation request to the system
Data storage server: The core of the storage system, stores data and responds to data operation requests from the application server
Management center server: Main-master composed of two machines A hot backup small-scale server cluster is responsible for cluster management, health heartbeat detection of data storage clusters; cluster expansion and fault recovery management; providing cluster address configuration information services to application servers, etc.
B. High availability solutions under different fault conditions
1. Fault classification of distributed storage systems: instantaneous fault, temporary fault, permanent fault
2. Transient fault resolution: multiple retries
3. Temporary fault resolution: manual intervention is required, the problematic server uses a temporary storage server
4. Permanent fault resolution: enable backup server replacement Permanently invalid server
12. Online shopping flash sale system architecture design case analysis
A. Technical challenges of flash sale activities:To the existing website business Impact, high concurrency applications, database load, sudden increase in network and server bandwidth, direct order placement
B. Response strategy for the flash sale system
Independent deployment of flash sale system
Static flash sale product page
Rent network bandwidth for flash sale activities
Dynamicly generate random order page URL
C. Flash sale system architecture design
1. How to control the lighting of the purchase button on the flash sale product page: use a JS file, modify the content at the beginning, request it every time, not be cached by CDN, etc., and use a random version number.
2. How to only allow the first submitted order to be sent to the order subsystem: control the entrance to the order page, so that only a few users can enter, and other users directly enter the flash sale end page. For example, there are 10 servers, each processing 10 requests. When the number of requests exceeds 10, the others will return errors, and then request the global cache record. If it is the first one, it will enter the order page, and the others will return failures.
13. Analysis of typical failure cases of large websites
A. Writing logs can also cause failures
The application's own log output configuration and the log output of third-party components must be configured separately
Check the log configuration file, let's play with the log. Michelle considers at least Warn
Need to turn off too many Error logs that some third-party components may output
B. High Faults caused by concurrent access to the database
The homepage should not access the database
The homepage should be static
C. Faults caused by locks under high concurrency conditions
Be careful when using lock operations
D. Cache Faults caused
The cache server is already an integral part of the website architecture and needs to be managed at the same level as the database
E. Faults caused by out-of-sync application startup
F. Faults caused by exclusive disk reading and writing of large files
Do not share storage for small files and large files
G .Faults caused by abusing the production environment
Be extra careful when accessing the production environment, and please have a dedicated DBA maintain the database
H.Faults caused by non-standard processes
Use the diff command to compare the code before submitting it to confirm that no code that should not be submitted is submitted; strengthen code review, have at least one other engineer conduct a code review before submission and share the responsibility for failures caused by the code
I. Failures caused by bad programming habits
Pay attention to the handling of empty objects, null values, etc.
14. Architect The art of leadership
A. Focus on people rather than products
1. A group of excellent people doing something they love will definitely achieve success
2. The best software management is to explore the outstanding potential of each member of the project team
3. Find a goal worth striving for together and create a job where everyone can maximize their self-worth. Atmosphere
B. Discover the excellence of people
1. Things make people, not people make things
2.Most people , including ourselves, we are all better than we think. Some excellence needs to be stimulated in the right environment, such as doing something challenging, cooperating with better people, or having the courage to surpass ourselves
3. Discovering the excellence of people is far more meaningful than discovering outstanding people
C. Sharing a beautiful blueprint
1. The blueprint should be clearly stated of: what the product should do, what it should not do, and what business goals it should achieve
2. The blueprint should be visual: what value can the product create for users, what market goals can it achieve, and what will the product look like in the end?
3. The blueprint should be simple: in one sentence Understand: What are we doing?
4. Architects should maintain focus on the target blueprint and be alert to any designs and decisions that deviate from the blueprint. Wrong deviations must be corrected in a timely manner. Necessary changes must be discussed by everyone, and Need to regain everyone's approval.
D. Jointly participate in the architecture
1. Don’t be the only architect who owns the architecture
2. Let others maintain the framework and architecture documentation
E. Learn to compromise
Opinions against architectural and technical solutions are essentially paying attention to, trying to understand and accept these solutions. Architects should not be too sensitive and should share their opinions frankly and seek common ground while reserving differences
2. Arguments about technical details should be verified immediately instead of continuing to discuss
3. When everyone is not discussing the architecture, make it clear Architecture has been integrated into projects, systems and developers. The sooner the architect is forgotten, the more successful the architecture is.
F. Achieve others
1. Our work Not only to produce products, but also to achieve people, and ultimately ourselves
2. To complete a project, we must not only create value for customers and make profits for the company, but also allow project members to grow
3. As the technical leader of the team, the architect should not try to control anything during the project process. Advance with a flexible plan and blueprint, and the team will take care of themselves
15. Website Architect Career Guide
#The purpose of developing software is to solve real-world problems, but many times people don’t know what the real problems are.
Many problems will also be encountered in the software development process. It is necessary to coordinate the interests of all parties to obtain the greatest possible support. It is necessary to balance customer needs, software output, and development. The relationship between resources requires many things to be done to realize the original blueprint of software design.
A. Discover problems and find breakthroughs
When expectations cannot be met, people will feel that something is wrong, because there is a problem It is the gap between experience and expectation. There are two ways to eliminate problems: improve the experience or lower expectations. Just lowering expectations will not solve the problem. On the contrary, facing the difference between expectations and actual experience can identify the problem and find a breakthrough.
2. The first thing new employees need to do is to integrate into the team
3. The last thing new employees need to do is to prove their abilities.
B. Ask questions and seek support
1. The problem is found, it is only the problem of the problem finder, not the problem owner, if you want To solve a problem, you must raise the issue and let the owner of the problem know that the problem exists.
2. Tips for raising questions:
Represent "my problem" as "our problem"
Ask closed-ended questions to your boss (give AB options for the boss to choose which one is better), and ask open-ended questions to your subordinates
Point out the problem instead of criticizing the person
Ask questions in an agreeable way
#3. The so-called outspokenness means that the intention you want to express must be stated directly. Understand, don’t talk in circles, but be careful in the way of expression, and take into account the feelings of the parties involved
C. Solve the problem and achieve performance
1. Solve my problem Before you solve the problem, solve your problem first
If you help others solve their problem, others will also help you solve their problem
In In the process of helping others solve problems, I became familiar with the situation
You use your solution to solve other people's problems, and this solution is under your control
2. Proper escape from problems
16. Manhua website architect
A. Divided by function Architect
Design architect, fire-fighting architect, evangelist architect, Geek architect
B.Divide architects by effect
Sherpa architect: usually develops the most technically difficult and challenging modules in the project, Spartan architect, dignitary architect
C. Press Division of Responsibilities and Roles Architect
Product Architect: Participate in the entire life cycle of the product, basic service architect (platform architect), infrastructure architect
D .Divide architects by level of concern
Architects who only focus on functions, architects who focus on non-functions, architects who focus on team organization and management, architects who focus on product operations, and architects who focus on the future of the product Architect
E. Divide architects by word of mouth
The best architect, a good architect, an average architect, a poor architect, and the worst Architects
F. Non-mainstream ways to divide architects
Ordinary architects, literary architects, 1 1 architects
Appendix A: Overview of large-scale website technologies
A. Front-end architecture
Browser optimization technology, CDN, static and dynamic separation, independent deployment of static resources, image service, Reflection proxy, DNS
B. Application layer architecture
Development framework, page rendering, load balancing, Session management, dynamic page staticization, business splitting, virtualized server
C. Service layer architecture
Distributed messaging, distributed services, distributed cache, distributed configuration
D. Storage layer architecture
Distributed files, relational databases, NoSQL databases, data synchronization
E.Backend architecture
Search engine, data warehouse, recommendation system
F.Data collection (log) and monitoring
Browser data collection, server business collection, server performance data collection, system monitoring, system alarm
G. Security architecture
Web attack , Data protection
H. Data center computer room architecture
Computer room, cabinet, server
The above is the detailed content of Mysql large website technical architecture core case analysis. For more information, please follow other related articles on the PHP Chinese website!