Abstract
The growing popularity of mobile and cloud computing raises new challenges related to energy efficiency. This work evaluates four various SQL and NoSQL database solutions in terms of energy efficiency. Namely, Cassandra, MongoDB, Redis, and MySQL are taken into consideration. This study measures energy efficiency of the chosen data storage solutions on a selected set of physical and virtual computing nodes by leveraging Intel RAPL (Running Average Power Limit) technology. Various database usage scenarios are considered in this evaluation including both local usage and remote offloading. Different workloads are benchmarked through the use of YCSB (Yahoo! Cloud Serving Benchmark) tool. Extensive experimental results show that (i) Redis and MongoDB are more efficient in energy consumption under most usage scenarios, (ii) remote offloading saves energy if the network latency is low and destination CPU is significantly more powerful, and (iii) computationally weaker CPUs may sometimes demonstrate higher energy efficiency in terms of J/ops. An energy efficiency measurement framework is proposed in order to evaluate and compare different database solutions based on the obtained experimental results.
1 Introduction
Energy efficiency is becoming increasingly important with the adoption of resource-constrained mobile and wearable devices (Pinto and Castor,
2017; Saxe,
2010). Furthermore, popularity of large-scale and distributed data centres hosting thousands of servers raises additional energy-related challenges (Chong
et al.,
2014; Owusu and Pattinson,
2012). Energy usage optimization techniques range from individual device sleeping state management to large-scale energy usage consolidation. Two main energy optimization goals can be highlighted.
Firstly, minimizing energy usage in order to prolong system operation is the main goal for wearable and mobile systems. Such systems typically aim to operate for prolonged time periods. Typical examples include mobile phones and smart watches. Given the miniature size, such systems suffer from low energy and computing power capacity. Despite the significant progress in CPU development and manufacturing, user needs commonly exceed the capacity of current hardware. Thus, apart from improving the underlying hardware, two techniques commonly used in conjunction are employed in order to improve overall user experience. These include idle state management and computation offloading. Both measures can significantly increase system operational time (Mansouri and Babar,
2021; Jiang
et al.,
2020).
Secondly, consolidating energy usage for multiple devices is applicable to data-centres. Perhaps the most popular approach is to virtualize multiple nodes and co-host them on the same hardware. Also, migrating virtual machines between hardware servers based on current workloads can be beneficial under certain conditions. The main goal of such dynamic VM migration is to pack the running tasks into a lower number of hardware servers. Consequently, this allows to power down (or transition to a low energy consumption state) the currently unused hardware. Two main obstacles in this area are the unpredictability of workload and large overheads associated with workload migration.
In this study, we focus on database-level workload offloading energy efficiency. We selected modern and popular databases including Cassandra, MongoDB, and Redis as NoSQL databases and MySQL as a relational database. We considered two different situations in regard to database server and client locations, namely same node or different nodes. We explore the impact of both CPU power and distance between client and database server nodes on the energy consumption. We made a setup of two physical nodes including a laptop and a powerful server and one virtual computing node managed through OpenStack virtualization platform. We further refer to the laptop and the powerful sever as mid-range and high-end CPU respectively, indicating that the server hardware we use is significantly more powerful than the laptop.
With the advent of NoSQL databases in last decade, a myriad of studies explored the performance of these databases, mainly including Cassandra, MongoDB, CouchDB, Riak, Redis and compared their performance with the well-known relational databases like MySQL, PostgreSQL, and SQL server. The evaluation of these databases was conducted on a variety of hardware infrastructures. Rabl
et al. (
2012) measured the performance of NoSQL and relational databases on a private cloud. Kuhlenkamp
et al. (
2014) performed experiments on the public cloud to evaluate how scalable and elastic NoSQL databases are. We have also recently measured the performance of NoSQL and relation databases on the hybrid cloud (Mansouri
et al.,
2020). The closest work to ours is energy measurement of three NoSQL databases (Cassandra, MongoDB, and Redis) running on a hardware server (Phung
et al.,
2019).
We focused on the evaluation of energy efficiency, taking into consideration the CPU power and distance between client and server through the following research questions (RQ).
-
• RQ1: Which database solution is more energy efficient under typical local usage?
-
• RQ2: How is the database energy efficiency affected by multiple threads?
-
• RQ3: In terms of energy efficiency, what is the role of CPU computational power in typical database workloads?
-
• RQ4: How beneficial is database workload offloading in terms of energy efficiency?
-
• RQ5: How does the network latency affect offloading energy efficiency?
To answer these questions, a set of experiments has been conducted in various conditions and environments. The main contributions of this study include insights on energy efficiency of the selected database under different scenarios as well as the goal-oriented database workload offloading energy efficiency evaluation and comparison framework.
The rest of this paper is organised as follows. A review of related literature is presented in Section
2. Section
3 provides a description of our infrastructure and experimental setup. Corresponding experiment results are outlined in Sections
4–
6. Section
7 discusses the proposed goal-oriented energy efficiency evaluation framework. Result summary and research limitations are discussed in Section
8. Lastly, Section
9 concludes this study and outlines future research directions.
2 Related Work
Energy measurement approaches can be categorized into
hardware-based monitoring and
software-based estimation. Hardware-based approaches require energy monitors which can be expensive depending on the precision provided. Such monitoring devices range from simple analogue Watt-meters to complex internet-connected systems. While costing more, smart monitoring systems enable data offloading to remote hosts for further analysis (Hackenberg
et al.,
2013; Hudlet and Schall,
2011). A detailed taxonomy of energy measurement approaches for battery-powered devices at various levels of granularity is presented in Ghaleb (
2019).
Software-based energy estimation approaches, though less precise, enable estimating the energy usage based on a pre-built power model for a given hardware. Such models are typically based on various performance counters and I/O load observable within the system (Rodríguez-Martínez
et al.,
2011; Kavanagh and Djemame,
2019; Rauber
et al.,
2014). Common examples include CPU and RAM frequency. Energy estimation models have been shown to be less precise compared to hardware-based measurement. The variance can achieve 73% to 300% (Fahad
et al.,
2019). Various metrics useful for practical software energy efficiency evaluation are discussed in Kalaitzoglou
et al. (
2014), Johann
et al. (
2012), Stier
et al. (
2015).
Intel developed a precise power usage model for modern CPUs (Khan,
2018,
2018). This model, known as RAPL (Running Average Power Limit), was developed to design and optimise cooling subsystem based on the observed power consumed and dissipated by a CPU (Zhang and Hoffman,
2015). Comparing the observed energy usage to RAPL measurements indicates that the overall results seem to match closely (Desrochers
et al.,
2016). Note that RAPL readouts are less accurate when a system is idle or when a discrete GPU is actively used. The overhead of RAPL measurement itself was measured in Calandrini
et al. (
2013). In addition, direct RAPL benchmarking tools integration have been proposed (Beyer and Wendler,
2020).
Due to the high popularity of virtualization, significant efforts have been put in VM energy usage estimation (Xu
et al.,
2015). Virtualization complicates the power usage monitoring as the hypervisor may not expose the performance counters to the guest OS. Thus, various estimation approaches have been proposed, including for nested virtualization scenarios (Colmant
et al.,
2015). It has been shown that Docker containerization is not more energy efficient compared to traditional VM hypervisors (Jiang
et al.,
2019).
More fine-grained techniques focus at the process-level energy usage estimation (Bourdon
et al.,
2013). Some prior research on energy efficiency of databases indicates that CPU power consumption may differ up to 60% depending on type of the operations (Tsirogiannis
et al.,
2010). This research was, however, limited to PostgreSQL and System-X databases only. More recent study, focused on NoSQL databases (HBase, Cassandra, HadoopDB (Abouzeid
et al.,
2009) and Hive), indicates that a significant portion of energy is wasted during waiting periods (Li
et al.,
2014). Zhou
et al. (
2020) measured the impact of number of CPU cores in the context of PostgreSQL energy efficiency. Database query-level energy efficiency and optimization is considered in Xu
et al. (
2012). Benchmarks in Schall
et al. (
2010), Cheong
et al. (
2012) indicate that using SSDs instead of mechanical platter HDDs improves database energy efficiency significantly. Furthermore, it has been demonstrated that the choice of the operating system as well as programming language may also affect database energy efficiency (Capra
et al.,
2012; Pereira
et al.,
2017).
In addition to measuring efforts, various database energy efficiency optimization techniques are proposed (Graefe,
2008). These include memory hierarchies, adaptive query rescheduling, I/O optimizations and data format improvements. Relying on hardware capabilities shows an improvement of database energy efficiency by up to 40% (Pisharath
et al.,
2004). Hardware-assisted energy-efficient database accelerator device has been presented in Haas
et al. (
2016). It was also shown that adjusting power usage limits on-the-fly within distributed systems may improve overall energy efficiency for a given workload (Phung
et al.,
2019). A similar approach proposes to disconnect and reconnect database nodes based on current processing needs to improve energy efficiency (Härder
et al.,
2011). Energy usage optimisation measures in large scale environments typically focus on optimisations such as power-aware resource management, task scheduling and traffic tuning (Jin
et al.,
2017).
In contrast to NoSQL solutions energy efficiency evaluation (Gomes
et al.,
2020), our study also considers a traditional SQL database. We also evaluate more usage scenarios, including close-proximity and long-distance database workload offloading. This study is an expansion of our prior work on data offloading energy efficiency evaluation (Prokhorenko and Babar,
2023).
3 Experiment Setup
This section outlines the details of the conducted energy efficiency measurement experiments. Firstly, our physical and virtual infrastructure is discussed in Section
3.1. Typical YCSB database workloads used throughout the experiments are described in Section
3.2. Energy usage measurement approach taken is presented in Section
3.3. Lastly, Section
3.4 outlines the conducted experiment details.
3.1 Hardware Infrastructure
Two physical nodes and one virtual node hosted on OpenStack cloud were used throughout the experiments. Ubuntu 18.04.4 server edition was used on all hosts. Table
1 summarises hosts details. Due to technical challenges associated with precise VM energy usage measurement, the virtual node was only used in workload offloading scenarios. In other words, the energy consumption was only considered from end-user perspective and measured on physical client nodes only. As clouds provide “infinite” resources (including energy supply) this limitation seems acceptable.
Table 1
Hardware infrastructure.
Host Name |
Type |
CPU |
RAM |
Mid-range |
Phys. |
Intel Core i5-5200U |
16 GB |
High-end |
Phys. |
Intel Core i9-9900KF |
32 GB |
OpenStack |
Virt. |
Intel Xeon E5-2623v3 |
8 GB |
Table 2
Database solutions.
Database |
Version |
Type |
Cassandra |
3.11.6 |
Wide column |
MongoDB |
3.6.3 |
Document-oriented |
MySQL |
5.7.29 |
Relational |
Redis |
4.0.9 |
Key-value |
3.2 Database Benchmark and Workloads
We conducted a number of experiments to determine energy efficiency of four data management solutions. We selected MongoDB, Cassandra, MySQL and Redis for this study due to their wide adoption in industry and academic sectors. This selection allows to compare a variety of different types of data management systems. The choice features two NoSQL, one traditional SQL and one RAM-based key-value database. Table
2 summarises database solutions benchmarked.
Table 3
YCSB workload descriptions.
Workload |
Workload type |
Description |
A |
Update heavy |
50% reads, 50% writes |
B |
Read mostly |
95% reads, 5% writes |
C |
Read only |
100% reads |
D |
Read latest |
Reading newly inserted records |
E |
Short ranges |
Querying short ranges of records |
F |
Read-modify-write |
Reading, modifying and writing back records |
We used Yahoo! Cloud Servicing Benchmark (YCSB) to run typical database workload as detailed in Table
3. As per YCSB recommendations, the following workload order was used: A, B, C, F, D are executed followed by purging and reloading data prior to running workload E. This order is recommended as workload D inserts new records, increasing the database size. The number of records and operations used throughout the local (scenarios 1–2) and close-proximity (1.5 ms latency) offloading scenarios was set to 100000 (scenarios 3–4). Long-distance (50 ms latency) offloading ran with the number of operations reduced to 10000 (scenarios 5–6).
3.3 Energy Usage Measurement
Intel RAPL technology was used to measure CPU energy usage throughout the experiments. In accordance with best practices, overall CPU micro-Joules measurements provided by Linux kernel was used to determine the amount of energy consumed by each workload. RAPL technology does not use a hardware meter but is rather a software power model. This power model uses various hardware performance counters and I/O models to approximate CPU energy usage. However, real-world energy usage measurements conducted by Intel indicate that the model closely matches actual power usage (Rotem
et al.,
2012). Note that power and energy have different meaning. Specifically, power is measured in Watts and indicates how much energy is used at any given moment. Energy is measured in Watt-hours and refers to the ability to deliver power over time period.
The initial intended purpose of RAPL was to measure how much power does a CPU need on average during a specified time period. This is an essential knowledge required to develop cooling systems for size-constrained devices. Knowing energy usage can also aid choosing batteries suitable for a given device. RAPL technology is available starting from Sandy Bridge generation on desktops and servers.
Table 4
Experimental scenarios.
Scenarios |
Source |
Destination |
Distance |
Concept |
Scenario 1 |
High-end CPU |
High-end CPU |
0 |
Local |
Scenario 2 |
Mid-range CPU |
Mid-range CPU |
0 |
Local |
Scenario 3 |
Mid-range CPU |
High-end CPU |
Close-proximity |
offloading |
Scenario 4 |
High-end CPU |
OpenStack VM |
Close-proximity |
offloading |
Scenario 5 |
Mid-range CPU |
High-end CPU |
Long |
offloading |
3.4 Database Usage Scenarios
Based on our hardware and virtual infrastructure, we ran six scenarios as described in Table
4. We run YCSB workload and databases locally on Mid-range CPU (scenario 1) and high-end CPU (scenario 2). We then offload workload from mid-range CPU to High-end CPU and OpenStack VMs, which are located in close proximity (scenarios 3 and 4). Finally, we run long-distance offloading from Mid-range and high-end CPU to OpenStack VMs (scenarios 5 and 6).
The results are classified in three groups: local, close-proximity offloading and long-distance offloading. Local refers to the scenario when both database server and client (YCSB) are located on the same host. Some databases (such as MySQL) may use local Unix Domain Sockets instead of network communication in such configurations by default. Close-proximity offloading was performed within Adelaide University LAN with the network latency of 1.5 ms between the nodes. Long-distance offloading was conducted over wider area network with the latency of 50 ms. Experiments were executed using different number of threads (1–20). Direct comparison of energy efficiency between different database solutions is not particularly meaningful due to the difference in architectures and data structures. Nevertheless, some high-level observations can be made based on the obtained results.
4 Workload Energy Efficiency Evaluation
This section focuses on database solution energy efficiency under typical local usage (RQ1), and on how is the database energy efficiency affected by multiple threads (RQ2). The first experiment set was conducted on a single host with no network communication and encryption overhead (scenarios 1–2).
![infor567_g001.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20524)
Fig. 1
Energy consumption of local processing on high-end CPU for Cassandra, MongoDB, MySQL and Redis (scenario 2).
4.1 High-End CPU
Energy usage per 1000 operations for a high-end CPU is depicted on Fig.
1. As can be seen from the presented graphs, Workloads B, C and D are quite similar in terms of energy efficiency. Namely, the relative database energy efficiency remains the same across workloads. The largest difference in energy efficiency exists for one-threaded experiment runs. This difference steadily drops as number of threads increases. At around 10 threads, the energy consumption stabilizes for all databases and does not improve any further.
In contrast to workloads B, C and D, for workloads A and F, the relative performance of MySQL and Cassandra is swapped for low number of threads. However, at 8 threads this difference diminishes. Consistent to previously conducted performance benchmarking (Mansouri
et al.,
2020), workload E proves to be most challenging for Redis with the other databases not being affected as much. Moreover, the energy efficiency of Redis does not improve much as number of threads increases, with no apparent changes after 2 threads. While the improvement between 1 and 2 threads is significant, Redis is worse more than twice than other databases. This is in contrast to other workloads, where Redis is twice as more energy-efficient.
Workload E is most demanding in terms of absolute values of energy usage as querying ranges of records requires more data to be transmitted per each operation. Larger transmissions naturally take longer to complete, thus requiring more energy. Lastly, the overall average thread-related energy usage improvement is a factor of 2 for all databases, except Redis.
4.2 Mid-Range CPU
Figure
2 illustrates a surprising result for the mid-range CPU. Namely, while achieving lower throughput, the slower CPU is more energy-efficient in terms of absolute values of micro-Joules consumed per 1000 operations. This remains true for all types of workloads with a roughly same relative database energy efficiency swap for Cassandra and MySQL can be observed for Workloads A/F vs. B-D. Somewhat lower energy usage improvement factor for increasing number of threads can be explained by the lower number of CPU cores available. Overall, the workload energy usage efficiency is consistent across the different CPUs in local usage scenarios.
![infor567_g002.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20525)
Fig. 2
Energy consumption of local processing on a mid-range CPU for Cassandra, MongoDB, MySQL and Redis.
Table 5
Energy usage efficiency of different workloads (A, B, C, D, and F) to Workload E.
Host name |
Database |
A |
B |
C |
D |
F |
Max variance |
Mid-range CPU |
Cassandra |
24% |
21.4% |
19.7% |
23% |
28% |
5.07 |
|
MongoDB |
42.3% |
35.8% |
35.4% |
31.6% |
54.1% |
3.16 |
|
MySQL |
37.8% |
28.1% |
26% |
28.5% |
47.7% |
3.84 |
|
Redis |
3.6% |
3% |
2.9% |
3.4% |
4% |
34.26 |
High-end CPU |
Cassandra |
24.1% |
23.6% |
23% |
23.6% |
29.5% |
4.35 |
|
MongoDB |
37.7% |
27.1% |
28.3% |
28.2% |
45.4% |
3.69 |
|
MySQL |
30.2% |
28.8% |
25.6% |
29.3% |
39.6% |
3.91 |
|
Redis |
3.8% |
3.4% |
2.8% |
2.8% |
4.4% |
35.78 |
Mid-range CPU → High-end CPU |
Cassandra |
33.3% |
30.1% |
27.7% |
32.6% |
38.3% |
3.61 |
MongoDB |
27.1% |
29.1% |
26.7% |
25% |
35.9% |
4 |
MySQL |
20.7% |
25% |
37.4% |
26.1% |
32.9% |
4.82 |
Redis |
4.4% |
3.7% |
3.4% |
4.2% |
6.4% |
29.49 |
4.3 Workload Type Effects
Table
5 summarizes cross-workload energy efficiency difference for workloads A, B, C, D, and F compared to the workload E. For example, it can be seen that Cassandra workload A uses 24% of energy used for workload E. As shown in Table
5 energy consumption difference can reach a factor of 35 for Redis across all databases and workloads. Omitting workload E, which is known to pose challenges for Redis, the difference ranges from a factor of 3.16 for MongoDB to 5.07 for Cassandra (last column in Table
5).
The variance between workloads is not the same across different CPUs and does not generally reduce when a more powerful CPU is used. For instance, workload-induced energy consumption variance is 3.16 for MongoDB on a mid-range CPU, whereas on a high-end CPU this variance increases to 3.69. Disregarding workload E, we see that with the exception of MySQL, workload F is least energy efficient for all databases across all configurations tested. Workload A is second hardest for local usage scenarios in terms of energy efficiency. For offloading scenarios, however, pattern changes for MongoDB and MySQL. Averaging the results across databases and usage scenarios, workloads can be sorted in terms of energy consumption as follows: E, F, A, D, B, C.
In summary, the answer to RQ1 is: Redis is more energy-efficient for most workloads, except workload E, for which MongoDB is the most efficient by a slight margin. The answer to RQ2 is: Increasing number of threads improves overall energy efficiency on average by a factor of 2, with no visible differences after number of threads reaches 10.
![infor567_g003.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20526)
Fig. 3
CPU power consumption of workload A for all databases, as well as idle and maximum utilization of (a) mid-range CPU, and (b) high-end CPU.
5 CPU-Related Energy Efficiency Evaluation
This section focuses on the role of CPU computational power from energy efficiency perspective in typical database workloads (RQ3). In our case slower CPU achieved higher energy efficiency. While the more powerful CPU can certainly complete the tasks faster by achieving a significantly higher ops/sec throughput, the slower CPUs consumes less energy per 1000 operations. Figures
3a and
3b demonstrate power consumption for mid-range and high-end CPUs, respectively.
We measure the idle CPU energy usage, with all databases and non-system processes stopped for one hour to establish idle usage baseline. We also capture the energy consumption of both CPUs under 100% utilization using the “stress” tool to determine maximum possible energy usage. For brevity, we plotted power usage for workload A for all databases. The following observations can be made based on the results.
First, we see that most of the databases do not fully utilize CPU maximum power. On a high-end CPU only MongoDB was able to achieve close to 100% power. Redis was also consuming a significant portion of maximum power. In contrast, MySQL and Cassandra could only consume around 15–25% and 30–60% on high-end and mid-range CPUs, respectively. These sub-optimal results indicate that there is still space in optimizing MySQL and Cassandra energy efficiencies. It is also somewhat unexpected to see the minor drops in energy usage for Cassandra and MySQL at 8 and 13 threads, respectively. A likely explanation is that this drop occurs due to the operation latency increasing as number of threads increases. In other words, after a certain load, databases do less useful work and spend more time waiting for the results.
Second, the key insight emerges from the comparison of the absolute values of idle to fully loaded energy usage of the CPUs tested. Specifically, mid-range CPU consumes 14.5 Watts, while high-end CPU consumes 119 Watts at full load. Therefore, under ideal conditions, the high-end CPU needs to perform 8.2 times more operations per second in order to achieve the same energy efficiency (J/ops) as the mid-range CPU. Under real-world conditions, due to the imperfection of database implementations this factor is reduced to around a factor of 4 for MySQL/Cassandra and a factor of 7 for Redis. While MongoDB could fully utilize the energy available for high-end CPU, it performed worse on mid-range CPU causing the equivalent performance factor to be closer to 10.
In summary, the answer to RQ3 is complex and depends on three key factors. Namely, the maximum power that a given CPU can consume, how much of that energy can a given database utilize and the raw computational CPU power (ops/sec). Precise answer is complicated by the fact that even the same database may utilize a different proportion of CPU power as algorithms may scale differently on different CPUs.
The mid-range CPU proved to be more energy-efficient than the high-end one under all workloads for all databases. However, the high-end CPU achieved a significantly higher throughput for all tasks. Therefore, actual CPU and database selection decision depends on the expected usage scenarios and system goals. Choosing a more energy-efficient CPU could be beneficial if a lower throughput is acceptable.
6 Network Offloading Energy Efficiency
This section discusses answers on how beneficial is database workload offloading in terms of energy efficiency (RQ4), and how does the network latency affect offloading energy efficiency (RQ5). Two sets of experiments were conducted to measure network effects. Firstly, a close-proximity offloading was performed within University of Adelaide LAN (Section
6.1). Then, a longer-distance offloading was conducted over WAN (Section
6.2). The energy measurements were only conducted on client side as servers are considered to have “infinite” power available unlike the resource-constrained client nodes. Such offloading is a common strategy in optimizing energy usage and overall system performance (Lu
et al.,
2020). Measuring energy usage on both client and server sides would most likely demonstrate a higher total energy consumption (compared to local processing) due to the data transmission overheads. Some research suggests that while offloading computations to a remote cloud could be beneficial for some workloads, extra factors such as data privacy and security must be considered (Kumar and Lu,
2010).
6.1 Close-Proximity Offloading
In this set of experiments, database servers resided on the high-end CPU, while YCSB was running on the mid-range CPU (Scenario 3). The same set of workloads was executed by YCSB and the total client-side energy consumption was measured. As shown in Fig.
4, the relative database energy efficiency is similar to local workloads. Redis struggles in workload E, while using less energy for other workloads.
![infor567_g004.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20527)
Fig. 4
Energy consumption of all databases, where database servers run on a high-end CPU and YCSB client runs on a mid-range CPU (close proximity).
![infor567_g005.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20528)
Fig. 5
Energy consumption of workload A for databases running on a mid-range CPU (local) vs. energy consumption of databases offloading from mid-range CPU to high-end CPU with distance of several meters (remote).
![infor567_g006.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20529)
Fig. 6
Energy consumption of workload A for databases running on a high-end CPU (local) vs. energy consumption of databases offloading from high-end CPU to OpenStack VM (remote).
Comparing the local and close-proximity offloading reveals that in some cases offloading can be beneficial in terms of energy efficiency. Figure
5 illustrates the difference between energy usage in local and close-proximity offloading scenarios. We compare local mid-range CPU workload execution and offloading from mid-range CPU to high-end CPU.
Offloading workloads to high-end CPU generally improves energy efficiency. However, for the effect to become visible and substantial for MongoDB, the number of threads must be increased. Offloading did not gain any significant improvement for Redis in terms of energy efficiency. Even at 20 threads the energy consumption difference remains negligible. For Cassandra, MongoDB and MySQL the energy efficiency improvement factor reaches 1.5, 1.9 and 2.7, respectively. Note that such significant energy usage improvement can only be achieved with multi-threaded loads as single-threaded workloads do not benefit from offloading as much.
Figure
6 shows the results of offloading from high-end CPU to a somewhat slower Openstack VM (Scenario 4). Packet round-trip time was 1.5ms for this set of experiments. This scenario reveals a drastic change compared to the mid-range to high-end offloading. Only MongoDB managed to achieve a slight improvement, reaching a factor of 1.3 at 20 threads. MySQL suffered most, consuming around 4 times more energy. Cassandra was only impacted slightly by the offloading with the energy consumption growing by around 10%. Redis consumed more than 50% more energy. The situation is worse for MongoDB and Cassandra when a low number of threads is used. For up to 6–7 threads, even close-proximity offloading to a slower CPU worsens energy efficiency.
These results are explained by the extra time that client idles while waiting for server results. The associated network traffic and bandwidth were measured and observed to be significantly lower than the physical link capacity. Thus, it is highly unlikely that network link limitations caused a bottleneck.
![infor567_g007.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20530)
Fig. 7
Energy consumption of databases, where YCSB runs on mid-range CPU and database servers run on high-end CPU (long distance).
6.2 Long-Distance Offloading
In this set of experiments, YCSB workload runs on the mid-range CPU and the high-end CPU hosts databases (Scenario 5). Compared to close-proximity offloading, the packet latency is significantly higher in this scenario (50 ms vs. 1.5 ms).
As shown in Fig.
7, measuring energy efficiency for long-distance workload offloading revealed that with an exception of workload E, the difference between different databases virtually disappears. Workload E is a known weak point for Redis. Other databases demonstrate close results with deviations rarely exceeding 10%. As average operation latency in local scenarios is typically around tens of microseconds, clearly 50 ms network latency dominates database latency.
Figure
8 illustrates the absolute values of energy usage for three scenarios: local processing on a mid-range CPU as well as close-proximity (LAN) and long distance (WAN) offloading from mid-range CPU to high-end CPU. We see that long-distance offloading worsens energy efficiency even if the remote CPU is significantly more powerful. Energy usage increases by an order of magnitude, making such offloading pointless. Results for MongoDB, MySQL and Redis are omitted for brevity due to similar result patterns.
Based on the obtained energy usage pattern measurements conducted, the answers to RQ4 and RQ5 can be formulated as follows. RQ4: workload offloading can be beneficial for some databases (all except Redis) only if the remote CPU is significantly more powerful in terms of computational capacity. RQ5: only low-latency network environments can provide benefits of workload offloading as high network packet latency starts dominating database-induced latencies. Based on these answers, a generic goal-based energy efficiency measurement and decisions approach is presented in the next section.
![infor567_g008.jpg](https://informatica.vu.lt/journal/INFORMATICA/article/1343/file/20531)
Fig. 8
A comparison of energy consumption of Cassandra (Workload A) running on a mid-range CPU between local and offloaded processing (close-proximity and long distance).
7 Goal-Based Database Energy Efficiency Evaluation Framework
A lightweight energy efficiency estimating framework has been developed as a part of this study. This framework enables automated database workload offloading energy efficiency measurement for given local and remote CPUs. YCSB-based workloads are used to obtain the measurements.
Network data transmission- and encryption-related energy usage overheads may be measured separately, if necessary. While specific conditions may lead to significant overheads, under the testing environment these overheads were negligible (around 1%). The proposed framework aids comparing a local host against a remote node in terms of energy efficiency.
Depending on the intended database usage and goals, energy-aware decisions can be made based on the obtained results. Two potential goals are considered: highest throughput and lowest energy usage. Different hardware and network environments may lead to these two goals becoming mutually exclusive (e.g. remote node may achieve higher throughput only at the expense of higher energy usage).
Table
6 shows an example output for the “high-end CPU → Openstack VM” evaluation. It is shown that running workload A for Cassandra is always more energy efficient and faster on the local node. In contrast, running workload B on MongoDB and MySQL the remote (Openstack) node becomes beneficial in terms of energy usage for 20 threads. Green cells highlight most energy efficient scenario for each database.
Table 6
The recommended energy-efficient (micro-Joules/1KOps) scenarios – coloured in green – for all databases running on high-end CPU (local) and offloading from high-end CPU to OpenStack VM (remote) for workloads A and B.
8 Applicability and Lessons Learnt
The most significant limitation of this study is that only CPU energy usage was considered. Taking into account RAM and disk I/O may affect our findings. Another constraint is the limited variety of hardware available, as all tested systems were Intel-based. Some prior measurements indicate that ARM may demonstrate higher energy efficiency (Abdurachmanov
et al.,
2014). Traditional x86-targeted energy efficiency optimizations were shown not to be suitable for mobile ARM-based database usage scenarios (Yang
et al.,
2014). A significant difference in energy efficiency has been observed for Intel CPUs of different generations (Lopez-Novoa,
2019). Lastly, only a subset of databases was considered.
Note that due to the limits of RAPL counters, longer workloads may cause energy usage readings to overflow returning negative results. Extra care must be taken in such cases to counter RAPL overflows. Powerful CPUs may overflow the RAPL counter in less than 30 minutes under load.
Redis exhibited stability issues with long-distance offloading. Multiple crashes occurred for Redis-related experiments causing interruptions. Extra care was taken to detect such crashes even during the data loading phase, as otherwise the experiments continued with an incomplete dataset. The high number of crashes forced to run Redis experiments with a reduced number of records. While Redis crashes could be attributed to low link quality, other databases worked correctly.
Additional measures were taken to minimize measurement noise. Clean operating systems were installed for all of the experiments and all non-essential services have been disabled. While this later proved to have an insignificant effect, all of the databases, except the currently tested one, were stopped to exclude possible background maintenance activities.
Compared to local runs, remote offloading faced several technical challenges. Due to firewall rules, SSH tunnelling was used in order to achieve inter-network offloading. SSH protocol involves encryption, which allows to secure the communication between database client and server nodes. Encryption, however, introduces overhead which may impact the energy efficiency. Thus, measurements were conducted to determine the network transmission and encryption overheads separately. Network transmission and encryption related energy consumption overheads were observed to be negligible compared to the database energy consumption and thus not taken into consideration. Actual energy consumption overhead was measured to be on average around 1% of the overall energy consumed during workload execution.
Interestingly, not all databases were able to achieve 100% consumption of the available CPU energy. In other words, stress-testing the CPU consumed significantly more energy than most of the databases did under load of 20 threads. This indicates that CPU architecture-specific optimizations may potentially improve database performance.
Idle energy usage measurements were also taken to determine the associated energy waste. The results indicate high-end and mid-range CPUs consume 9 and 1.6 Watts, respectively. Thus, to be more energy-efficient, the high-end CPU needs to achieve significantly higher throughput.
9 Conclusion and Future Work
It is challenging to provide generalized results in the context of energy efficiency due to the large amount of different factors involved. However, a number of interesting quantitative results have been obtained throughout this research study. It was shown that newer CPUs, while being faster, may not necessarily be more energy efficient in all workloads. We also see that most of the tested databases were unable to utilize the amount of energy consumed under stress testing. As energy efficiency can be improved by a factor of 3 to 4, the area of software optimizations is worth investigating.
Energy efficiency improvements were only achieved under two conditions: (i) The remote CPU is significantly more computationally powerful than the local CPU, and (ii) the network latency is low. When either condition was not fulfilled, the energy efficiency decreased dramatically. Deploying workload offloading in practice requires preliminary estimations of hardware capacity, workload type and network latency. Unsuitable resources and conditions mean that performing the computations locally would be more energy efficient. This is explained by the high overhead of two-way data transmission and the associated idle waits. Note that the performed measurements revealed that data transmission energy usage is negligible (1.2–1.5%) compared to the idling energy waste. Running an energy usage estimation benchmark in a given environment prior to making a choice is thus highly recommended. Extending the applicability of the offloading approach to clustered database environments and dynamically moving client nodes would require dynamic offloading feasibility assessment.
Future research lines can focus on a number of potentially useful extensions of this work. Firstly, testing more CPUs (AMD and ARM). Secondly, tweaking memory management related settings such as swap file size, disk I/O caching and amount of memory used by a database might reveal further differences between databases tested.
Thirdly, individual hardware components energy usage can be measured separately. More fine-grained results could be obtained by considering different types of disk storage, such as SSD and spinning hard disks of different speeds.
Fourthly, database-tailored workloads can be executed to estimate the effect of different data storage architectures. For example, document-based storage might demonstrate improved energy efficiency for shorter documents. Similarly, a relational database may perform differently depending on the record fields number.
Lastly, OS-specific effects could also impact energy efficiency of databases. Such difference might not directly relate to the database quality but rather indicate different OS-specific disk I/O caching approaches. This knowledge would be useful in making informed decisions on optimal database selection in terms of energy efficiency.