Abiquo Documentation Cookies Policy

Our Documentation website uses cookies to improve your experience. Please visit our Cookie Policy page for more information about cookies and how we use them.


Abiquo 4.7

Skip to end of metadata
Go to start of metadata

Introduction

This is a basic overview of the Abiquo accounting system:

  1. Abiquo records when users reserve or use resources in the kinton_accounting DB.
  2. The administrator configures which resources are accounted and the accounting periods used for it.
  3. Some accounting procedures are run periodically to consolidate/update resource usage and purge old records.
 Accounting overview diagram

Accounting information backup

The accounting master tables are in kinton_accounting DB. Include these tables in your backup procedures:

  • accounting_event_ips
  • accounting_event_pm
  • accounting_event_repository
  • accounting_event_storage
  • accounting_event_vlan
  • accounting_event_vm
  • accounting_event_ds_storage
  • accounting_event_vm_on
  • accounting_event_vm_off
  • accounting_event_backup
  • accounting_event_firewall
  • accounting_event_loadbalancer

Accounting parameters

To retrieve the current accounting parameters, run the query below in kinton_accounting DB:

SELECT * FROM accounting_parameters;
 See the command output ...
+---------+-------------------------------------+------------+
| idParam | paramName                           | paramValue |
+---------+-------------------------------------+------------+
|       1 | AccountingEnabled                   | 1          |
|       2 | DeleteRegEventsUseSPParam           | 0          |
|       3 | DeleteRegEventsDeleteHours          | 26280      |
|       4 | MaximumPeriodsToFirstInit           | 1          |
|       5 | MaximumPeriodsToProcess             | 24         |
|       6 | AccountPeriodSize                   | HOUR       |
|       7 | AccountPeriodGranularity            | HOUR       |
|       8 | LoggingLevel                        | INFO       |
|       9 | VirtualMachine-vcpu                 | 1          |
|      10 | VirtualMachine-vram                 | 1          |
|      11 | VirtualMachine-vhd                  | 1          |
|      12 | VirtualMachine-vtierhd              | 1          |
|      13 | VirtualMachine-backup               | 1          |
|      14 | ExternalStorage                     | 1          |
|      15 | IPAddress                           | 1          |
|      16 | VLAN                                | 1          |
|      17 | VirtualMachine-hypervisorType       | 1          |
|      18 | Firewall                            | 1          |
|      19 | LoadBalancer                        | 1          |
|      20 | VirtualMachine-vcpuon               | 1          |
|      21 | VirtualMachine-vcpuoff              | 1          |
|      22 | VirtualMachine-vramon               | 1          |
|      23 | VirtualMachine-vramoff              | 1          |
|      24 | VirtualMachine-hp                   | 1          |
|      25 | VirtualMachine-hpon                 | 1          |
|      26 | VirtualMachine-hpoff                | 1          |
|      27 | VirtualMachine-haHosted             | 0          |
|      28 | ReservedPhysicalMachine-cpu         | 0          |
|      29 | ReservedPhysicalMachine-ram         | 0          |
|      30 | RepositoryStorage                   | 0          |
|      31 | VirtualMachine-antiAffinity         | 0          |
|      32 | Consolidation-time-sensitivity-secs | 30         |
+---------+-------------------------------------+------------+

To check the value of an accounting parameter, use the query below:

SELECT * FROM accounting_parameters WHERE paramName="PARAM_NAME";
 Example...

SELECT * FROM accounting_parameters WHERE paramName="VirtualMachine-antiAffinity";
+---------+-----------------------------+------------+
| idParam | paramName                   | paramValue |
+---------+-----------------------------+------------+
|      20 | VirtualMachine-antiAffinity | 0          |
+---------+-----------------------------+------------+
1 row in set (0.00 sec)


Description of resources

Abiquo stores this data about the use of these resources in the Accounting events tables. Then Administrators can configure which resources are accounted in the Accounting event detail table.

Resource

Cat.

Description

Unit

Billing Type

Association

Private cloudPublic cloud

Cores

1

vCPUs in VMs created by users

vCPU

Consumption

VM

(tick)(tick)

Cores ON

15

vCPUs in VMs deployed and running

vCPU

Consumption

VM

(tick)(tick)

Cores OFF

16

vCPUs in VMs deployed and stopped

vCPU

Consumption

VM

(tick)(tick) 

RAM

2

RAM in VMs created by users

Megabytes

Consumption

VM

(tick)(tick)

RAM ON

17

RAM in VMs deployed and running

Megabytes

Consumption

VM

(tick)(tick) 

RAM OFF

18

RAM in VMs deployed and stopped

Megabytes

Consumption

VM

(tick)(tick) 
Local Storage3

Disk space connected to the hypervisor as datastores for VM disks,
including secondary hard disks

Bytes

Consumption

VM

(tick)(tick)

External Storage

4

Disk space for use as external volumes created by users on VMs

Bytes

Reservation

VM/Independent

(tick) 

Public IPs

5

IP addresses with Internet access for use by VMs

No. of IP addresses

Reservation

VM/Independent

(tick)(tick)

VLANs

6

Private VLANs for VMs

  • Unmanaged, external and public network VLANs are not accounted
  • A private VLAN is automatically created when a user creates a VDC

No. of VLANs

Reservation

VDC

(tick)(tick)
Firewall19Firewalls created by the userNo. of FirewallsReservationVDC(tick)(tick) 
Load Balancer20Load balancers created by the userNo. of Load BalancersReservationVDC(tick)(tick) 

Hypervisor Type

7

The hypervisor type for the VM

HV type

Consumption

VM

(tick)(tick)

Cost Code


Optional cost codes for virtual image templates

Cost code

Consumption

VM

(tick)(tick)
Backup14Backup policies applied to VMs deployedBackup policyConsumptionVM(tick) 
Hardware profile21VM Hardware profileHardware profileConsumptionVM(tick) (tick)
Hardware profile ON22VM Hardware profile in the ON stateHardware profileConsumptionVM(tick) (tick)
Hardware profile OFF23VM Hardware profile in the OFF stateHardware profileConsumptionVM(tick) (tick)
HA Hosted VM8VM deployed on a high-availability rackTrue or falseConsumptionVM(tick) 
Reserved Physical
Machine Cores
9CPU of each server that has been explicitly reserved for an enterprise.
Recorded even if the server is not being used.
vCPUReservationEnterprise(tick)(minus)
Reserved Physical
Machine RAM
10RAM of each server that has been explicitly reserved for an enterprise.
Recorded even if the server is not being used.
MegabytesReservationEnterprise(tick)(minus)
Repository Storage11

Repository space is used when:

  • Customers create a VM instance
  • Customers upload disk images into the template repository
  • Abiquo converts instances and templates to other hypervisor formats

Abiquo accounts for space used by instances and templates (cases 1 and 2).
If you wish to account conversions, create a custom metric (case 3).

Bytes used by each
template
ConsumptionEnterprise(tick) 
Anti-affinity VM12VM deployed in an anti-affinity layer separate from other VMs in the same layerTrue or falseConsumptionVM(tick) 
Tiered local
datastore storage
13Local storage on hypervisor datastores by datastore tiers (including primary and secondary
non-persistent hard disks) 
Bytes used by each diskConsumptionVM(tick)  
Custom
See Custom Accounting Resource DefinitionsUser definedUser definedUser defined(tick)

(tick)


Key

  • (tick) - Resource accounted

  • (minus)  - Deprecated
  • Blue shading - disabled by default
  • Pink shading - custom resource


Enable or disable accounting of resources

Abiquo always tracks all resources in the accounting master tables, but Administrators can configure which resources will be consolidated for billing and reporting in the accounting_event_detail table.

To select which resources Abiquo will record in this table, enable/disable the corresponding accounting parameters by setting the corresponding accounting parameter value.

 Available parameters ...


Parameter Name

Default Value

VirtualMachine-vcpu

1

VirtualMachine-vcpuon1
VirtualMachine-vcpuoff1
VirtualMachine-vram

1

VirtualMachine-vramon

1

VirtualMachine-vramoff

1

VirtualMachine-vhd

1

ExternalStorage

1

IPAddress

1

VLAN

1

Firewall1
LoadBalancer1
VirtualMachine-hypervisorType

1

VirtualMachine-backup1

VirtualMachine-haHosted

0

ReservedPhysicalMachine-cpu

0

ReservedPhysicalMachine-ram

0

RepositoryStorage

0

VirtualMachine-antiAffinity0
VirtualMachine-tierhd0


 Example: Enable resource accounting ...

To enable VirtualMachine-antiAffinity accounting for VMs deployed in anti-affinity layers, update its paramValue to 1:

UPDATE accounting_parameters SET paramValue=1 WHERE paramName="VirtualMachine-antiAffinity";

Accounting time parameters

In the accounting_event_detail table, Abiquo stores resource usage information by account period size and account period granularity. Always set these parameters before allowing users to work with your system.

Account period size parameter

For each accounting period, the update procedure checks the Accounting event tables for resources that have been reserved and/or used in the last period by granularity and records the usage data in the accounting_event_detail table.

The minimum period of consumption of a resource is always one accounting period. If a resource is active for an interval of less than one accounting period, the system automatically rounds consumption up to one accounting period. But please note that the value stored will depend on the granularity.

Resources may be turned on and off during an accounting period. Abiquo stores all these events in the Accounting master tables. However, accounting_event_detail only contains the maximum value for an accounting period.

You can set the accounting period size parameter in the accounting_parameters table.

ParameterDefault valuePossible valuesDescription
AccountPeriodSizeHOUR

HOUR

DAY

WEEK

MONTH

Controls the size of the aggregated account period of the accounting_event_detail table.

If the value is bad or missing, the default value of 1 hour is used.


 Example: Change the accounting period to WEEK ...
UPDATE accounting_parameters SET paramValue="WEEK" WHERE paramName="AccountPeriodSize";

Accounting period granularity

For each accounting period, the Update procedure checks the master tables for resources reserved and/or consumed in the last period by granularity  and records the usage data in the Accounting event detail table.

The value stored for each accounting period can be divided into smaller time units using the granularity. However, the granularity does not affect the minimum usage.

The minimum period of usage is always determined by the accounting period:

  • If accounting period = DAY and granularity = HOUR, then for a single CPU VM, Abiquo would record 24 resource units for a day record. So this could represent a charge for 24 hours of vCPU usage.
  • If accounting period = DAY and granularity = DAY, then Abiquo will record 1 resource unit. So this could represent a charge for 1 day of vCPU usage.

You can set the granularity parameter in the accounting_parameters table.


Default valuePossible valuesDescription
AccountPeriodGranularityHOUR

HOUR

DAY

WEEK

MONTH

Granularity of the resource units within an accounting period.

If the value is bad or missing, the default value is used.

The granularity must be smaller than the accounting period and the accounting period must be divisible by the granularity.

The following table shows compatibles periods and granularities. Abiquo uses calendar months, so week granularity is not compatible with month.


 


Period

Hour

Day

Week

Month

Granularity

 

 

 

Hour

(tick)

(tick)

(tick)

(tick)

Day

 

(tick)

(tick)

(tick)

Week

 

 

(tick)

 

Month

 

 

 

(tick)

 Example: Set the accounting period granularity to DAY

We assume that the account period size is set to WEEK:

UPDATE accounting_parameters SET paramValue="DAY" WHERE paramName="AccountPeriodGranularity";

Check the updated value by using the select command.

Change the accounting period parameters in an existing system

To change the accounting period, follow these steps: 

  1. Backup the accounting_event_detail table.
  2. Delete the existing content of the accounting_event_detail table.
  3. Change the accounting parameters to the desired values. Set MaximumPeriodsToFirstInit to ensure old data is accounted correctly.
  4. Monitor the content of accounting_event_detail to ensure the generated data meets your needs. If not, then repeat from step 2.

Accounting periodic procedures

The accounting system runs the accounting procedures below periodically to:

  • UpdateAccounting processes the data in the accounting master tables to consolidate the resource usage in the accounting_event_detail table.
  • DeleteOldRegisteredEvents deletes data in the accounting master tables older than the DeleteRegEventsDeleteHours accounting parameter (26280 hours or 3 years by default)

These procedures are configured in the /etc/cron.d/abiquo-accounting cron job. Note that DeleteOldRegisteredEvents does not use its legacy parameter to set the maximum record age by default.

To view the cron job configuration for the accounting procedures, run the command:

cat /etc/cron.d/abiquo-accounting  
 See the command output ...
# Runs every hour
0 * * * * root mysql -uroot -hlocalhost -e "CALL kinton.UpdateAccounting();"
# Runs every Sunday at 12:00 and deletes records older than a week.
0 12 * * 0 root mysql -uroot -hlocalhost -e "CALL kinton.DeleteOldRegisteredEvents(168);"

Set the frequency of procedure run to delete old records

To change the maximum record age of the accounting master table records, set the DeleteRegEventsDeleteHours parameter in accounting_parameters table as desired.

It is also possible to use the stored procedure parameter as maximum record age by enabling the DeleteRegEventsUseSPParam parameter in accounting parameters, but it is not recommended to use this method.

 Example: Set the record maximum age to two years...:
UPDATE accounting_parameters SET paramValue="17520" WHERE paramName="DeleteRegEventsDeleteHours";


Retrieve Accounting data

It is important to avoid running queries on the accounting_event_detail table. Queries on this table may negatively affect the Abiquo Server.

You should replicate the accounting data to allow user access through accounting queries. See Kinton Database replication for further information.

Use the following queries as templates for creating queries to view aggregate resource usage as required. These queries should retrieve information from the replicated accounting database.

Previously the accounting system used views, which have been maintained for legacy applications. See Abiquo accounting architecture.

 Hourly Usage Sum Template Query ...
Template for query to view HOURLY USAGE SUM
SELECT
  v.startTime AS startTime,
  v.endTime AS endTime,
  v.idAccountingResourceType AS idAccountingResourceType,
  v.resourceType AS resourceType,
  SUM(v.resourceUnits) AS resourceUnits,
  v.idEnterprise AS idEnterprise,
  v.idVirtualDataCenter AS idVirtualDataCenter,
  v.enterpriseName AS enterpriseName,
  v.virtualDataCenter AS virtualDataCenter
FROM
  kinton_accounting.HOURLY_USAGE_MAX_VW v
WHERE
  # APPLY CUSTOM QUERY FILTERS HERE, E.G. BY DATE RANGE AS BELOW
  v.startTime >= '2012-04-16' AND v.endTime < '2012-04-17'
GROUP BY
  v.startTime,
  v.idAccountingResourceType,
  v.idEnterprise, v.idVirtualDataCenter;
  Daily Usage Sum Template Query ...
Template for Query to view DAILY USAGE SUM
SELECT
  CAST(a.startTime AS DATE) AS startTime,
  CAST(a.startTime AS DATE) AS endTime,
  a.idAccountingResourceType AS idAccountingResourceType,
  a.resourceType AS resourceType,
  SUM(a.resourceUnits) AS resourceUnits,
  a.idEnterprise AS idEnterprise,
  a.idVirtualDataCenter AS idVirtualDataCenter,
  a.enterpriseName AS enterpriseName,
  a.virtualDataCenter AS virtualDataCenter
FROM
  kinton_accounting.HOURLY_USAGE_MAX_VW a
WHERE
  # APPLY CUSTOM QUERY FILTERS HERE, E.G. BY DATE RANGE AS BELOW
  a.startTime >= '2012-04-16' AND a.endTime < '2012-04-17'
GROUP BY
  CAST(a.startTime AS DATE),
  a.idAccountingResourceType,
  a.idEnterprise, a.idVirtualDataCenter;
  Monthly Usage Sum Template Query ...
Template for query to view MONTHLY USAGE SUM
SELECT
  CAST((a.startTime - INTERVAL (DAYOFMONTH(a.startTime) - 1) day) AS DATE) AS startTime,
  LAST_DAY(a.startTime) AS endTime,
  a.idAccountingResourceType AS idAccountingResourceType,
  a.resourceType AS resourceType,
  SUM(a.resourceUnits) AS resourceUnits,
  a.idEnterprise AS idEnterprise,
  a.idVirtualDataCenter AS idVirtualDataCenter,
  a.enterpriseName AS enterpriseName,
  a.virtualDataCenter AS virtualDataCenter
FROM
  kinton_accounting.HOURLY_USAGE_MAX_VW a
WHERE
  # APPLY CUSTOM QUERY FILTERS HERE, E.G. BY DATE RANGE AS BELOW
  a.startTime >= '2012-04-01' AND a.endTime < '2012-05-01'
GROUP BY
  CAST((a.startTime - INTERVAL (dayofmonth(a.startTime) - 1) day) AS DATE),
  a.idAccountingResourceType,
  a.idEnterprise, a.idVirtualDataCenter;

Accounting consistency checks

You can use the Abiquo accounting consistency check to monitor your accounting system.

See Abiquo accounting consistency check