Abiquo Documentation Cookies Policy

Our Documentation website uses cookies to improve your experience. Please visit our Cookie Policy page for more information about cookies and how we use them.


Abiquo 4.7

Skip to end of metadata
Go to start of metadata


Introduction to accounting architecture

The Abiquo accounting architecture consists of accounting tables, triggers for data capture, and procedures.

This detailed diagram shows the architecture of the Accounting system.

 

Triggers and stored procedures

The Abiquo kinton schema contains triggers for accounting integration. The triggers run stored procedures in the kinton_accounting schema to automatically capture data in if/then/when scenarios.

ResourceTriggers
VMs
update_virtualmachine_update_stats AFTER UPDATE ON virtualmachine
Storage
create_rasd_management_update_stats AFTER INSERT ON rasd_management
delete_rasd_management_update_stats AFTER DELETE ON rasd_management
IPs
update_ip_pool_management_update_stats AFTER UPDATE ON ip_pool_management
update_rasd_management_update_stats AFTER UPDATE ON rasd_management
VLANs
create_vlan_network_update_stats AFTER INSERT ON vlan_network
delete_vlan_network_update_stats AFTER DELETE ON vlan_network
Racks
rack_update AFTER UPDATE ON rack
Physical Machines
physicalmachine_create AFTER INSERT ON physicalmachine
physicalmachine_delete AFTER DELETE ON physicalmachine
physicalmachine_update AFTER UPDATE ON physicalmachine
VM templates
virtualimage_create AFTER INSERT ON virtualimage
virtualimage_delete AFTER DELETE ON virtualimage
virtualimage_update AFTER UPDATE ON virtualimage
virtualimageconversion_update AFTER UPDATE ON virtualimage_conversion


Stored procedures for event accounting

The triggers collect the information necessary and call stored procedures responsible for recording accounting events. The stored procedures are:

AccountingVMRegisterEvents
AccountingStorageRegisterEvents
AccountingIPsRegisterEvents
AccountingVLANRegisterEvents

Accounting tables

The accounting_event_"resource" tables are the definitive source of accounting data and log of customer activity. You must back up these tables to prevent accidental loss of accounting data. The default for removing stale data from these tables is 3 years. If you upgraded to Abiquo 2.4 and you were using default values, they were updated to 3 years. However, non-default values were not updated in the upgrade to Abiquo 2.4.

Accounting triggers are always active. The resultant data is stored in event-based database tables that store resource data, and the time resource usage begins and ends.

These tables all follow a similar design pattern and are prefixed accounting_event_"resource", where "resource" can be substituted depending on the accounting resource type. For example, the data table on virtual machines (CPU, memory, storage) shown below.


Table accounting_event_vm

 

Field Name

Field Type

Description

idVMAccountingEvent

BIGINT

ID event unique

idVM

INTEGER

ID of virtual machine owning the resource

idEnterprise

INTEGER

ID of Enterprise resource owner

idVirtualDataCenter

INTEGER

ID of Virtual Datacenter facility owner

idVirtualApp

INTEGER

ID Virtual App resource owner

cpu

INTEGER

'cores' units

ram

INTEGER

Memory, in megabytes (MB)

hd

BIGINT

Local storage, in bytes (B)

startTime

TIMESTAMP

When a resource was created

stopTime

TIMESTAMP

When a resources was destroyed

consolidated

TINYINT

Flag for processed or not

costCode

TINYINT

Cost code assigned for pricing purposes

hypervisorType

VARCHAR

The type of hypervisor hosting the virtual machine

idDataCenterINTEGERID of Datacenter
VirtualMachine-haHostedINTEGERFlag for hosted on HA-enabled PM or not

 

Table accounting_event_pm

This table stores details of reserved physical servers - one row for each reserved server.

 

Field NameField TypeDescription
idPMAccountingEventBIGINTUnique column identifier for the accounting event
idPhysicalMachineINTID of the physical machine, as defined in the kinton.physicalmachine table.
NOTE: Use the kinton_accounting ABQ_PM_ID_TO_NAME(id) function to obtain the machine name associated with the ID
idEnterpriseINTID of the ENTERPRISE, as defined in the kinton.enterprise table.
NOTE: Use the kinton_accounting ABQ_ENT_ID_TO_NAME(id) function to obtain the machine name associated with the ID
cpuINTThe number of physical CPUs in the machine
ramINTThe physical machine’s RAM (in MB)
startTimeTIMESTAMPThe time at which the machine reservation started
stopTimeTIMESTAMPThe time at which the reservation finished.   This value is NULL if the reservation is still active
idDataCenterINTID of the physical datacenter, as defined in the kinton.datacenter table.
NOTE: Use the kinton_accounting ABQ_DC_ID_TO_NAME(id) function to obtain the datacenter name associated with the ID
consolidatedTINYINTIndicates whether this row has been consoliated with other rows.   Currently this value is always 0
version_cINTSoftware controlled data version

 

Table accounting_event_repository

This table contains details of the repository usage. There is one row for each VM Template and Instance stored in the repository. Persistent VM templates are not included in this table, because they are stored as external volumes and charged separately.

 

Field NameField TypeDescription
idRepoAccountingEventBIGINTUnique column identifier for the accounting event
idImageINTID of the repository image, as defined in the kinton.virtualimage table.
NOTE: Use the kinton_accounting ABQ_IMG_ID_TO_NAME(id) function to obtain the image name associated with the ID
idEnterpriseINTID of the ENTERPRISE, as defined in the kinton.enterprise table.
NOTE: Use the kinton_accounting ABQ_ENT_ID_TO_NAME(id) function to obtain the machine name associated with the ID  
idImageTypeNameINTThe ID of the image type name in the generic accounting name table, use the kinton_accounting ABQ_OBJECT_ID_TO_NAME (id)  function to retrieve the actual type name.   Names are one of:
INSTANCE, TEMPLATE, INSTANCE-CONVERSION, TEMPLATE-CONVERSION
idImageFormatNameINTThe ID of the image format name in the generic accounting name table, use the kinton_accounting ABQ_OBJECT_ID_TO_NAME (id)  function to retrieve the actual format name.   Names are: VMDK_FLAT, RAW, etc. see the wiki API reference for further details
imageSizeBIGINTPhysical Size of the repository image, in bytes
idRepositoryINTThe ID of the image repository as defined in the kinton.repository table
idRepositoryName (minus)INTThe ID of the repository name in the generic accounting name table, use the kinton_accounting ABQ_OBJECT_ID_TO_NAME (id)  function to retrieve the actual repository name. THIS FIELD WAS DEPRECATED IN ABIQUO 3.4
startTimeTIMESTAMPThe time at which the image was created in the repository
stopTimeTIMESTAMPThe time at which the image was deleted from the repository.  This value is NULL if the image still exists.
idDataCenterINTID of the physical datacenter, as defined in the kinton.datacenter table.
NOTE: Use the kinton_accounting ABQ_DC_ID_TO_NAME(id) function to obtain the datacenter name associated with the ID.  
consolidatedTINYINTIndicates whether this row has been consoliated with other rows.   Currently this value is always 0.
version_cINTSoftware controlled data version.

accounting_event_detail table

The accounting_event_detail table contains the information needed for accounting and billing functions. This table should not be accessed directly - if you need to access non-aggregated data then please use the kinton_accounting schema ACCOUNT_PERIOD_USAGE_VW (or the HOURLY_USAGE_MAX_VW or HOURLY_USAGE_MAX_2_VW) views to access the content of this table.

Warning: Running unoptimized or long SQL queries against this table can have a significant impact on the overall performance of the transactional database, and might affect the proper functioning of AbiquoServer. Therefore Abiquo recommends the use of database replication to replicate this table to another database server which has been prepared to process OLAP type queries.

This table may accumulate significant amounts of information over time, and therefore its space requires suitable management.

The following rows are returned from ACCOUNT_PERIOD_USAGE_VW view:

Field Name

Field Type

Description

idVM

INTEGER

ID virtual machine owning the resource

idEnterprise

INTEGER

ID Enterprise resource owner

idVirtualDataCenter

INTEGER

ID Virtual Datacenter owner

idVirtualApp

INTEGER

ID Virtual App resource owner

enterpriseName

VARCHAR

Company Name resource owner

virtualDataCenter

VARCHAR

Virtual Datacenter name resource owner

virtualApp

VARCHAR

Virtual App Name resource owner

virtualmachine

VARCHAR

Virtual Machine Name resource owner

starttime

TIMESTAMP

Start of the time slice

endTime

TIMESTAMP

End of the time slice

idAccountingResourceType

TINYINT

Resource type posted: virtualmachine-vcpu, virtualmachine-vram, virtualmachine-vhd, ExternalStorage, IPAddress, VLAN, VirtualMachine-hypervisorType

resourceType

VARCHAR

Resource type (text)

resourceUnits

BIGINT

Resource units to account for

resourceName

VARCHAR

Resource Name to account for. A new resource type 'VirtualMachine-hypervisorType' is now recorded for each VM. The Hypervisor Type is recorded in this 'resourceName' column.

costCode

VARCHAR

Cost code of Virtual Image. Only valid when the idAccountingResourceType is 1 (virtualmachine-vcpu)

idStorageTier

INTEGER

Code associated with this storage resource's QoS/Tier level. Only valid for the 'External Storage' resource type

Every hour, the status of each resource implemented in the system is recorded, which can generate a large number of rows. Abiquo recommends that the table is periodically purged of resource information which is no longer required.

Resource IDs

The resources in the accounting_event_detail table are represented with a numerical value, as defined in the table below in the idAccountingResourceType column.

Parameter Name

idAccountingResourceType

VirtualMachine-vcpu

1

VirtualMachine-vram

2

VirtualMachine-vhd

3

ExternalStorage

4

IPAddress

5

VLAN

6

VirtualMachine-hypervisorType

7

VirtualMachine-haHosted

8

ReservedPhysicalMachine-cpu

9

ReservedPhysicalMachine-ram

10

RepositoryStorage

11

VirtualMachine-antiAffinity12
VirtualMachine-tierhd13

Resource owners

Abiquo records resource ownership at the following levels:

  • virtual machine
  • virtual datacenter 
  • enterprise

Description of resource owners

1. Virtual Machine

  • Cores
  • RAM
  • Local storage
  • Cost codes (if the VM template has a cost code)

    Grouped virtual machine components and virtual machines are accounted per group, not per individual virtual machine.

2. Virtual Appliance

  • Groups of virtual machines
    • Cores
    • RAM
    • Local storage
    • Cost codes (if the VM template has a cost code)

      Grouped virtual appliances are accounted per group, not per individual virtual appliance.

3. Virtual Datacenter

  • Groups of virtual machines
  • Groups of virtual appliances
    • Cores
    • RAM
    • Local storage
    • VLANS
    • Public IPs
    • External storage

      Virtual Datacenter accounting is the total of the resources reserved and/or consumed by the Virtual Machines, Virtual Appliances, and users of a Virtual Datacenter.

4. Enterprise
An Enterprise has no managed resources that do not belong to a Virtual Datacenter, so there is no accounting per Enterprise. However, the Enterprise associated with each resource is recorded for aggregating resources at Enterprise level, and this information can be retrieved through the database views.

 

Legacy accounting views

For accounting and billing, Abiquo has previously implemented database views (in both the kinton and kinton_accounting schemas) to help you access the resource usage information correctly. However, the use of these views is no longer recommended for scalability and performance reasons. Instead, use the equivalent queries described above.

ACCOUNT_PERIOD_USAGE_VW - Returns the resource use for the accounting period with the maximum values. If the same resource has been used more than once in the same time zone, it takes the greater value. Not grouped by virtual machine and virtual appliance. The columns returned ar the same as the HOURLY_USAGE_MAX_2_VW but the name has been changed to reflect the change to the accounting period.

Legacy maximum usage views

  • HOURLY_USAGE_MAX_VW - Returns the resource use for hours with their maximum values. If the same resource has been used more than once in the same time zone, it takes the greater value. Not grouped by virtual machine and virtual appliance.
  • HOURLY_USAGE_MAX_2_VW

Legacy sum views

  • HOURLY_USAGE_SUM_VW - Returns the resource use for hours with their values added together. The data is aggregated to virtual datacenter level, but can be further aggregated to enterprise level if required.
  • DAILY_USAGE_SUM_VW - Returns the resource use for days with their values added together. The data is aggregated to virtual datacenter level, but can be further aggregated to enterprise level if required.
  • MONTHLY_USAGE_SUM_VW - Returns the resource use for months with their values added together. The data is aggregated to virtual datacenter level, but can be further aggregated to enterprise level if required.

Configuration view

  • ACCOUNTING_CONFIG_VW  - Returns the current accounting configuration as a single row

 

  • No labels