Lakehouse-Powered Payment Risk at Scale
DOI:
https://doi.org/10.5281/zenodo.20537385Keywords:
Payment Risk Scoring Systems, Cloud-Native Big Data Architecture, Real-Time Risk Evaluation, Payment Service Providers, Financial Transaction Analytics, Machine Learning–Based Risk Metrics, Near-Native Response Time, Data Generation And Ingestion, Event-Driven Processing, Batch Data Processing, Cloud-Native IoT Systems, Risk Scoring Services, Lifelong Machine Learning, CI/CD Pipelines For ML, Data Governance And Privacy, Zero-Trust Security Architecture, Observability And Monitoring, Scalable Financial Platforms, Payment Analytics Data Lakes, Intelligent Financial Risk Management.Abstract
The growing volumes of payment transactions have placed significant burden on payment service providers or financial institutions to evaluate the risk profiles of customers swiftly. Various risk-related metrics of customers can be obtained from the data related to their payment transactions, and the recent development of machine learning-based modeling approaches allows these metrics to be modeled as a risk score. However, the shoe has on the other foot as providing such services has become a competitive game between players leading to the demand for near-native response time in serving customer requests. The services, therefore, require a cloud-native big data architecture where the data generation, ingestion, processing and modeling operations can be carried out at huge scales and low response and lead times while integrating the required principles of data governance and privacy. Cloud-native IoT systems collect vast volumes of data daily that forms the data lake for processing. Data processing includes both batch and event-driven processing. Batch processing is used for generating risk-related metrics where as event-driven processing is used for carrying out risk scoring. The ML models behind risk metrics are provided as services and lifelong learning is done by using CI/CD pipelines. Firewall and observability adopt a zero-trust policy to promote not just security but also privacy and governance compliance. Robust architecture has enormous readability and scalability features for tackling such diverse services. The entire architecture is illustrated with an example in the domain of payment service providers using a risk scoring system proposed in the previous work.
References
[1] Breidbach, C. F., Keating, B. W., & Lim, C. (2020). Fintech service ecosystems: Value co-creation in open banking. Journal of Service Management, 31(3), 395–417.
[2] Bughin, J., Seong, J., Manyika, J., Chui, M., & Joshi, R. (2018). Notes from the AI frontier: Modeling the impact of AI on the world economy. McKinsey Global Institute.
[3] Chen, M., Mao, S., & Liu, Y. (2014). Big data: A survey. Mobile Networks and Applications, 19(2), 171–209.
[4] Chen, Y., Wang, Y., Sui, Y., & Zhang, Z. (2022). Real-time fraud detection for digital payments using ensemble learning. Knowledge-Based Systems, 236, 107741.
[5] Dastile, X., Celik, T., & Potsane, M. (2020). Statistical and machine learning models in credit card fraud detection. Applied Soft Computing, 91, 106263.
[6] Deng, X., Huang, Z., & Cheng, X. (2019). Financial fraud detection using graph neural networks. IEEE Intelligent Systems, 34(4), 41–50.
[7] Eloranta, V., Ardolino, M., Saccani, N., & Adrodegari, F. (2021). From data to value: Digital servitization in platforms. Industrial Marketing Management, 97, 38–53.
[8] Fan, W., & Bifet, A. (2013). Mining big data: Current status and forecast. ACM SIGKDD Explorations Newsletter, 14(2), 1–5.
[9] Ferreira, J. J., Fernandes, C. I., & Ferreira, F. A. (2019). To be or not to be digital, that is the question. Journal of Business Research, 101, 583–590.
[10] Florez-Lopez, R., & Ramon-Jeronimo, J. M. (2015). Enhancing accuracy in fraud detection. International Journal of Accounting Information Systems, 17, 26–37.
[11] Garcia, S., Luengo, J., & Herrera, F. (2016). Data preprocessing in data mining. Springer.
[12] He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.
[13] Huang, Y., Wu, X., & Wang, Z. (2023). Cloud-native MLOps architectures for real-time AI systems. Journal of Cloud Computing, 12(1), 1–19.
[14] Jablonski, S., & Meiler, C. (2021). Workflow management in data-intensive cloud applications. Future Generation Computer Systems, 115, 582–594.
[15] Jain, A., Duin, R. P. W., & Mao, J. (2000). Statistical pattern recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 4–37.
[16] Kearns, M., & Roth, A. (2020). The ethical algorithm. Oxford University Press.
[17] Kim, Y., Cho, S., & Kim, J. (2022). Payment fraud detection using deep representation learning. Expert Systems with Applications, 195, 116569.
[18] Kleppmann, M. (2017). Designing data-intensive applications. O’Reilly Media.
[19] Kotsiantis, S., Zaharakis, I., & Pintelas, P. (2006). Machine learning: A review of classification and combining techniques. Artificial Intelligence Review, 26(3), 159–190.
[20] Kou, Y., Lu, C. T., Sirwongwattana, S., & Huang, Y. P. (2004). Survey of fraud detection techniques. IEEE International Conference on Networking, Sensing and Control, 749–754.
[21] Li, J., Xu, L., & Zhao, K. (2021). Privacy-preserving analytics for financial data in cloud platforms. IEEE Transactions on Cloud Computing, 9(3), 1234–1246.
[22] Liu, Y., Chen, X., & Ward, R. (2020). Risk-aware decision systems in digital finance. Decision Support Systems, 135, 113340.
[23] Lopez-Rojas, E., Elmir, A., & Axelsson, S. (2016). PaySim: A simulated credit card transaction dataset. Proceedings of the European Modeling and Simulation Symposium, 249–255.
[24] Mishra, D., Gunasekaran, A., Papadopoulos, T., & Dubey, R. (2018). Supply chain performance and big data analytics. International Journal of Production Economics, 195, 192–204.
[25] Mitchell, T. M. (1997). Machine learning. McGraw-Hill.
[26] Moreno, A., Serrano, M., & Fernández-Caballero, A. (2022). Explainable AI in financial risk management. AI & Society, 37(4), 1535–1549.
[27] Müller, R., & Jensen, O. (2019). Big data architectures for financial analytics. Journal of Big Data, 6(1), 1–18.
[28] Ngai, E. W. T., Hu, Y., Wong, Y. H., Chen, Y., & Sun, X. (2011). The application of data mining techniques in financial fraud detection. Decision Support Systems, 50(3), 559–569.
[29] Oliveira, T., Thomas, M., Baptista, G., & Campos, F. (2016). Mobile payment adoption. Computers in Human Behavior, 61, 404–414.
[30] Pasquale, F. (2015). The black box society. Harvard University Press.
[31] Rajesh, R. (2020). Analytics-driven decision making in financial services. International Journal of Information Management, 54, 102135.
[32] Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding machine learning. Cambridge University Press.
[33] Singh, A., Thakur, N., & Sharma, A. (2016). A review of supervised machine learning algorithms. International Journal of Computer Applications, 95(25), 17–24.
[34] Sun, Z., Wang, H., & Li, Y. (2023). Scalable stream processing for fraud detection in payment systems. Future Generation Computer Systems, 139, 179–191.
[35] Zhang, Y., Xiong, Y., & Zhou, W. (2024). Cloud-based intelligent risk scoring for digital payments. IEEE Transactions on Services Computing, 17(2), 456–469.
Additional Files
Published
Data Availability Statement
None
Issue
Section
License
This work is licensed under a Creative Commons
Attribution 4.0 International License (CC BY 4.0).
You are free to:
- Share: copy and redistribute the material
- Adapt: remix, transform, and build upon the material
for any purpose, even commercially.
Under the following terms:
Attribution — You must give appropriate credit to
the original author(s) and source.
Full license text: https://creativecommons.org/licenses/by/4.0/
