Introduction
An information technology audit is defined by Pathak (2005) as “the process of collecting and evaluating evidence to determine whether an information system safeguards assets, maintains data integrity, consumes resources efficiently, and achieves organisational goals effectively.” In Malaysia, according to Mahzan and Veerankutty (2011) 16% of audit firms perform IT audit assurance services. This trend towards IT audit is likely to increase as IT audit continues to be stressed by the accounting profession and the auditing standards. For example, International Standards of Auditing (ISA 315) requires auditors to examine and ascertain the IT procedures, processes, and controls when assessing the client’s controls environment.
IT has changed the way an audit is conducted (Kim et al. 2009) and has become pervasive in the way businesses conduct their operations and record their transactions. Organisations are increasingly relying on IT internal controls to safeguard their physical assets and data. For example, IT corporate governance framework, such as COBIT has provided guidelines and procedures on governance of IT in an organisation. The primary role of an IT audit is to ensure the integrity of an organisation’s information systems (Senft and Gallegos, 2008). IT audits are important organisational processes that add value to the organisation because they support the auditor’s judgement on the integrity, reliability and quality of the information produced by the organisation’s information systems (Gallegos et al. 2004).
To ensure the value of IT audits, organisations should implement a standard method for evaluating the quality of audits (Senft and Gallegos, 2008). There has been a stream of research into factors that make up IT audit quality (Havelka and Merhout 2007, Merhout and Havelka 2008, Stoel 2012, Havelka 2013). IT audit quality literature was developed from two strands of research. The first strand of research and the earliest IT audit quality framework study was using the inductive approach. A focus group was used to obtain and develop factors related to IT audit quality (Havelka and Merhout 2007). As this study consists of only a few respondents, it is likely that the factors generated could be biased. Havelka and Merhout expanded their study further by including more participants to validate their list (Merhout and Havelka 2008). Nevertheless, their numbers of participants are still low. Their framework called ITAP contains general categories and does not provide questionnaire tools. . The Merhout and Havelka study was later combined with research from Stoel et al (2012) that includes 140 indicators, 26 concepts and 6 broad categories. Thus, the framework is difficult to apply as a survey instrument.
The second strand of IT audit quality approach uses the deductive approach. Stoel et al. (2012) is the only known paper that developed and empirically tested a survey tool on IT audit quality. They developed the constructs by combining constructs from financial audit quality literature (Carcello, Hermanson et al. 1992, Vehn, Carcello et al. 1997) with the IT audit literature (Merhout and Havelka 2008). That paper then used factor analysis from the ICASA respondents from the United States to rank which factors have the most impact on IT audit quality. However, that research did not look at the effects that these factors have on the outcome of IT audit nor look at the relationships between these factors.
The purpose of this study is to extend previous research by developing a broad IT quality survey that is practical that could be used to understand the factors that affect IT audit quality. This paper will present preliminary descriptive results of those factors.
Literature Review and Theoretical Paradigms
Information technology (IT) has become an indispensable tool in modern business. The increased reliance on IT and the complex, evolving nature of IT systems, has resulted in the need to implement internal controls to safeguard commercial information (Stoel and Muhanna, 2011). Not surprisingly, IT in all of its facets, has received extensive scholarly attention. Recent attention on IT audit research has focused on the need for, and the conduct of, an IT audit (Al Omari, Barnes, and Pittman, 2013). To be classified as an IT audit, the examination must involve information technology, either as the specific focus of the examination (even indirectly, such as IT governance), or as the means to complete an engagement (Chong and Tan, 2012; Merhout and Halveka, 2008).
The extant literature on IT auditing centers either on the managerial decision to ‘reduce risk’ within a corporate governance framework (Al Omari, Barnes, and Pittman, 2012; Parkinson and Baker, 2005), or conceptualisations surrounding the conduct of IT audits. For instance, scholars have examined the strategy and standards of IT audits (Pealrson, 2001), computer assisted tools/software used in IT audits (Gallegos, Vlosky, and Vlosky, 1992; Gillevet, 1995), the planning (Lam, 2001) and management (Van Grembergen and De Haes, 2005) of the audit. From a broader perspective, scholars have studied the effectiveness of IT audits (Alzeban and Gwilliam, 2012), the role of an IT auditor (Chaney and Kim, 2007), the types of IT audits (Senft and Gallegos, 2008), the IT audit process (Gallegos, 2002) and the training of IT auditors (Curtis et al. 2009).
One area in the IT literature that has received little scholarly attention is the quality of IT audits. This is surprising given the increasingly critical function that IT plays in organisations, the need for a clearer understanding of what constitutes quality in IT auditing is needed. Consequently, there is no definition of IT audit quality. Regardless, any notion of IT audit quality is nestled in an organisation’s IT governance (Ferguson et al. 2013).
Model Framework
This section defines the IT audit quality constructs identified from the literature and also explains the associated items that reflect these constructs. We developed the dependent variable IT audit quality while the independent variable constructs are taken from Stoel, Havelka, and Merhout (2012) and Halveka and Merhout (2013). With a few exceptions, we use the same items as Stoel, Havelka, and Merhout (2012) while reclassifying a few items to the appropriate constructs. We also added items from Heroux (2012) and Havelka and Merhout (2013).
We posit that the following IT related factors; “Auditor IT Knowledge and Competencies”, “Internal Control Knowledge”, “Target System Complexity”, and “Resources” are correlated with IT Audit Quality while controlling for accounting variables that are found to affect audit quality as mentioned by Carcello et al. (1992), Samelson et al. (2006) and Vehn et al. (1997). The following sections describe in detail the constructs and scales used.
Fig 1: Proposed IT audit framework
IT Audit Quality
Although IT audit quality is not explicitly defined, it could be implied from the objectives of IT audit. We use the Delone and McLean (2003) paradigm that states that quality itself has measurable domain. IT audit quality is multidimensional and as such we propose that IT audit quality has the following dimensions;
- effectiveness that is whether IT audit could assess that the organisation information system is able to meet organisational goals (Weber 1988, Merhout and Havelka 2008),
- reliability that is how reliable is the IT audit conducted on the auditee (Stoel 2012),
- efficiency that is how well the IT audit is able to perform while minimising the cost (Stoel 2012),
- overall perception of quality that is how the IT audit is viewed (Lowensohn, Johnson et al. 2007).
Audit team’s IT knowledge and competencies
To be able to detect material weaknesses in IT systems, audit teams need to have knowledge and competencies not only about accounting and auditing, but also on IT specialised knowledge. Specialised IT professional qualifications and certifications have been shown to have more likelihood of involvement with auditing on IT governance, risks and controls (Héroux and Fortin, 2012). Knowledge about IT and accounting system, are shown to be important factors in IT audit quality (Havelka and Merhout, 2013; Stoel, Havelka, and Merhout, 2012). IT audit teams require knowledge of tools and techniques to help them audit “through the computer” rather “around the computer” (Janvrin, Bierstaker et al. 2008, Janvrin, Lowe et al. 2008). Lastly, understanding of the risks involved from the technology used was identified as a factor to IT audit quality (Havelka 2013). Accordingly, we took the construct and the following scales from Stoel (2012)
- specialised IT professional qualifications and certifications,
- knowledge of IT and accounting system,
- knowledge of CAAT (Computer-assisted auditing tools),
- knowledge of risks associated with technology use.
Audit team’s Internal Control Knowledge
Poor internal controls are likely to cause material misstatements in the financial statements (Ge and McVay 2005). Thus, knowledge of internal controls is an important facet of IT auditing (Stoel, Havelka, and Merhout, 2012). Specific IT internal controls have been found to have material impact on the quality of information produced (Li, Peters et al. 2010). The components of this specific IT internal control can be broadly divided into; information security, data processing integrity, and data structure controls (Li, Peters et al. 2010, Steinbart 2012). Information security covers any internal control that helps to protect the organisation’s data (Li, Peters et al. 2010). On the other hand, data processing integrity helps control reliability and accuracy of the data (Li, Peters et al. 2010). Data structure controls are about how well the data have consistent format (Li, Peters et al. 2010). Accordingly, we took the construct and the following scales from Stoel (2012)
- knowledge of internal controls,
- knowledge of information security,
- knowledge of data processing integrity,
- knowledge of data structure controls
Target system complexity
This construct refers to how difficult it is to audit the auditee (Havelka and Merhout, 2013). For this questionnaire we incorporated “Business Scale and Audit Scope” and “Auditability” into target system complexity because the construct is the function of business size and scale of the auditee, how broad the scope of the audit, the support given by the auditee and the reliability of the internal controls (Stoel, Havelka, and Merhout, 2012). These are then indicated by the following items provided by Stoel (2012):
- number of geographical dispersed business units,
- number of business units or processes or systems involved,
- support by auditee,
- how well the internal control is defined and documented.
Resources
Resources refer to the availability of audit
tools, time, budget and audit staff that the audit team could command to assist their IT auditing activities (Stoel, Havelka, and Merhout, 2012). These items include:
- whether computer-assisted auditing tools (CAATs) are used (Stoel 2012),
- whether there is enough time to conduct the IT audit (Héroux 2012),
- whether there is enough budget available to conduct the IT audit (Héroux 2012),
- whether there is enough staff to properly conduct the IT audit (Havelka 2013).
Results and Discussions
Organisation profile
About three quarters (74%) of the respondents are large companies and the rest medium size companies. This is consistent with the population because we surveyed only the Top Market Cap of the Malaysian Stock Exchange. As shown in Table 2, the top 3 industries our respondents are in; the construction industry (17%), manufacturing (16%) and food and hospitality (11%).
Table 1: Firm Size
Table 2: Industry of the respondents
Dependent and independent variables
Our dependent variable is the IT audit quality. The respondents rate IT audit performed in their organisation as effective, reliable and efficient (Table 3). For the independent variables (Table 3 to Table 7), target system complexity is ranked the highest (mean of 5.7) followed by internal control knowledge (mean of 5.65), and IT knowledge and competencies (mean of 4.74).
Table 3: IT audit quality
Table 4: Auditor IT Knowledge and Competencies
Table 5: Internal Control Knowledge
Table 6: Target System Complexity
Table 7: Resources
The correlation matrix as shown in Table 8 indicates that IT knowledge and competencies are moderately positively correlated with IT audit quality (p<0.001). All the other independent variables are shown to be significantly (p<0.05) positively correlated with IT audit quality except auditor independence and target system complexity. Nevertheless, since our sample size is small with increase in the number of respondents, the significance would likely increase as well. Our results seem to indicate that IT knowledge and competencies are important in determining the IT audit quality.
Table 8: Correlation Matrix (n=46)
There is also significant interaction between the independent variables. Firstly, audit planning and methodology is strongly correlated with auditor and audit interaction (p<0.001). Secondly, resources provided in the firm are also strongly correlated with auditor’s IT knowledge and competencies (p<0.001).
Conclusion and Discussion
IT has become pervasive and critical in successful operations and management of any organisations. Thus, it has become necessary to audit the information systems of organisations. Previous research has focused on questionnaires that have too many items and may not be practical. In addition, the factors affecting IT audit quality also have not been studied.
The contribution of this exploratory paper is to identify broad constructs from the literature that affects IT audit and used it to develop a questionnaire. Furthermore, this paper presents the preliminary descriptive statistics on the relationships of the factors that affect IT audit quality. We found that auditor’s IT knowledge and competencies are significantly correlated with IT audit quality. This has implication on policymakers and professional accounting bodies in improving IT audit quality.
Our research is limited by the number of respondents. This affects the statistical methods and inferences that can be drawn from the data. Nevertheless, our paper sets the stage towards more research in this area. The questionnaire used needs to be rigorously tested and validated.
Acknowledgement
This research is funded by the Ministry of Education of Malaysia under the Fundamental Research Grant Scheme (FRGS/2/2013/SS05/MUSM/02/4). We would also thank reviewers who have made numerous comments to improve this paper.
(adsbygoogle = window.adsbygoogle || []).push({});
References
Alzeban, A. and Gwilliam, D. (2012), ‘Perceptions of managers and internal auditors as to factors affecting the effectiveness of internal audit in the public sector context,’ Proceedings of the 10th European Academic Conference on Internal Audit and Corporate Governance, University of Verona, Italy.
Al Omari, L., Barnes, PH. and Pitman, G. (2012), ‘An exploratory study into audit challenges in IT governance: a Delphi approach,’ Proceedings of the Symposium on IT Governance, Management and Audit (SIGMA2012), Universiti Tenaga Nasional, Malaysia.
Al Omari, L., Barnes, PH. and Pitman, G. (2013), ‘A Delphi study into the audit challenges of IT governance in the Australian public sector,’ Electronic Journal of Computer Science and Information Technology, 4(1).
Carcello, JV., Hermanson, RH. and McGrath, NT. (1992), ‘Audit quality attributes: The perceptions of audit partners, preparers, and financial statement users,’ Auditing, 11(1), 1-15.
Chaney, C. and Kim, G. (2007), ‘The Integrated Auditor: All internal auditors need to understand core IT control concepts and risks to provide assurance in today’s technology-based business world,’ Internal Auditor, 64 (4), 46-52.
Chong, JL. and Tan, FB. (2012), ‘IT governance in collaborative networks: A socio-technical perspective,’ Pacific Asia Journal of the Association for Information Systems, 4 (2).
Curtis, MB., Jenkins, JG., Bedard, JC. and Deis, DR. (2009), ‘Auditors’ training and proficiency in information systems: a research synthesis,’ Journal of information systems, 23 (1), 79-96.
DeAngelo, LE. (1981), ‘Auditor independence, ‘low balling’, and disclosure regulation,’ Journal of Accounting and Economics, 3 (2), 113-127.
Delone, WH. and McLean, ER. (2003), ‘The DeLone and McLean model of information systems success: a ten-year update,’ Journal of Management Information Systems, 19 (4), 9-30.
Ferguson, C., Green, P., Vaswani, R. and Wu, GH. (2013), ‘Determinants of effective information technology governance,’ International Journal of Auditing, 17 (1), 75-99.
Gallegos, F. (2002), ‘The Audit Report and Follow-up: Methods and Techniques for Communicating Audit Findings and Recommendations,’ Information Systems Control Journal, 4, 17-20.
Gallegos, F., Klosky, JM. and Klosky, V. (1992), ‘Auditing Decision Support Systems: An Approach,’ South East Decision Sciences Conference Proceedings, Savannah, GA, Spring, pp. 161–164.
Ge, W. and McVay, S. (2005), ‘The disclosure of material weaknesses in internal control after the Sarbanes-Oxley Act,’ Accounting Horizons, 19(3), 137-158.
Gillevet, J. (1995), ‘Utilizing CAATs to determine the possibility of input errors in automated
Systems,’ IS Audit Control Journal, 4, 17–24.
Havelka, D. and Merhout, JW. (2007), ‘Development of an information technology audit process quality framework,’ AMCIS 2007 Proceedings, 61.
Havelka, D. and Merhout, JW. (2013), ‘Internal information technology audit process quality: Theory development using structured group processes,’ International Journal of Accounting Information Systems, 14(3), 165-192.
Héroux, S. and Fortin, A. (2012), ‘The internal audit function in information technology governance: A holistic perspective,’ Journal of Information Systems, 27 (1), 189-217.
International Federation of Accountants (IFAC). (2015). International Standard on Auditing 315 (revised) Identifying and Assessing the Risks of Material Misstatement Through Understanding the Entity and Its Environment.
Gallegos, F., Senft, S., Manson, DP. and Gonzales, C. (2004) Information Technology Control and Audit, (2nd ed.) Auerbach Publications, CRC Press, Boca Raton, FL.
Kim, HJ., Mannino, M. and Nieschwietz, RJ. (2009), ‘Information technology acceptance in the internal audit profession: Impact of technology features and complexity,’ International Journal of Accounting Information Systems, 10(4), 214-228.
Lam, J. (2001), ‘Top ten requirements for operational risk management,’ Risk Management, 48 (11), 1.
Mahzan, N., and Veerankutty, F. (2011), ‘IT auditing activities of public sector auditors in Malaysia,’ African Journal of Business Management, 5(5), 1551–1563.
Merhout, JW., and Havelka, D. (2008), ‘Information technology auditing: A value-added IT governance partnership between IT management and audit,’ Communications of the Association for Information Systems, 23(1), 26.
Parkinson, M. and Baker, N. (2005), ‘IT and enterprise governance,’ Information Systems Control Journal, 3, 17-21.
Pathak, J. (2005) Information technology auditing: an evolving agenda, Springer Science and Business Media.
Pearlson, KE. (2001) Management and Using Information Systems—A Strategic Approach, Wiley, New York.
Samelson, D., Lowensohn, S. and Johnson, LE. (2006), ‘The determinants of perceived audit quality and auditee satisfaction in local government,’ Journal of Public Budgeting, Accounting and Financial Management, 18 (2), 139-166.
Senft, S. and Gallegos, F. (2008) Information technology control and audit, CRC Press.
Steinbart, PJ., Raschke, RL., Gal, G. and Dilla, WN. (2012), ‘The relationship between internal audit and information security: An exploratory investigation,’ International Journal of Accounting Information Systems, 13 (3), 228-243.
Stoel, MD. and Muhanna, WA. (2011), ‘IT internal control weaknesses and firm performance: An organisational liability lens,’ International Journal of Accounting Information Systems, 12(4), 280-304.
Stoel, D., Havelka, D., and Merhout, JW. (2012), ‘An analysis of attributes that impact information technology audit quality: a study of IT and financial audit practitioners,’ International Journal of Accounting Information Systems, 13 (1), 60-79.
Van Grembergen, W., and De Haes, S. (2005). Measuring and improving IT governance through the balanced scorecard. Information Systems Control Journal, 2 (1), 35-42.
